How to find the current project id of the deployed Python function in Google Cloud gives error

Question

I have deployed a Python 3.7 function in Google Cloud. I need to get the project-id through code to find out where it is deployed.

I write a small Python 3.7 script and test it through Google shell command line

import urllib
import urllib.request
url="http://metadata.google.internal/computeMetadata/v1/project/project-id"
x=urllib.request.urlopen(url)
with x as response:
 x.read()

Unfortunately this gives me only b'' as response. I am not getting the project id though I have set it using

gcloud config set project my-project

I am new to Google Cloud and Python.

Issue 2

This is an additional issue below:

In my local system I have installed gcloud and if I run the above python3.7 script from there:

x=urllib.request.urlopen(url) #this line

I am getting this exception from the line above:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1350, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1240, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1286, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1235, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1006, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 946, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 917, in connect
    self.sock = self._create_connection(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 787, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

The below changes as suggested by the answer also gives me empty string:

Sepulchral

Answer 1

In the Python 3.7 runtime, you can get the project ID via an environment variable:

import os
project_id = os.environ['GCP_PROJECT']

In future runtimes, this environment variable will be unavailable, and you'll need to get the project ID from the metadata server:

import urllib.request
url = "http://metadata.google.internal/computeMetadata/v1/project/project-id"
req = urllib.request.Request(url)
req.add_header("Metadata-Flavor", "Google")
project_id = urllib.request.urlopen(req).read().decode()

Running this locally will produce an error because your local machine can't resolve the http://metadata.google.internal URL -- this is only available to deployed functions.

Tolan

Hi Thanks for this, I ran this at cloud shell, but the project_id is coming as empty string " " >>> import urllib.request >>> url = "metadata.google.internal/computeMetadata/v1/project/project-id" >>> req = urllib.request.Request(url) >>> req.add_header("Metadata-Flavor", "Google") >>> project_id = urllib.request.urlopen(req).read().decode() >>> project_id '' >>> print(project_id) >>> – Sepulchral

As mentioned by @Dustin Ingram, it will not work locally or in Cloud Shell, it is only available for deployed functions. I add an another answer giving an alternative, – Gnosticize

And please not that you are getting an empty string because you are not replacing projec-id from the url to one's of yours. – Gnosticize

@Nibrass H yes i understood thank you, but my goal is to find the project id which i have set to currently and not the default project id, so will this metadata-api find the current project id without me giving it in the url? metadata.google.internal/computeMetadata/v1/project/ – Sepulchral

It's correct that this won't work in Cloud Shell, but you don't need to modify the URL. This is an endpoint for determining the current project ID. – Tolan

Thanks @DustinIngram using this metadata url if a user wants to get the project id dynamically during function execution, is it not possible, using thie api url as the api requires a project-id parameter by itself – Sepulchral

No, it doesn't. The example works exactly as shown. – Tolan

not to zombie this comment though @DustinIngram but was the reasoning behind why this change was made discussed? also are older in 3.7 before going to become RESERVED words? – Ankh

Answer 2

In Cloud Shell, the DEVSHELL_PROJECT_ID environment variable contains your project ID.

So you can run the following command in order to get your project ID with Python.

import os
USER = os.getenv('DEVSHELL_PROJECT_ID')
print(USER)

And if you want to get your project ID in a Cloud Function, you can get it from the service account credentials.json which contains the project_id:

# If this is running in a cloud function, then GCP_PROJECT should be defined
if 'GCP_PROJECT' in os.environ:
    project_id = os.environ['GCP_PROJECT']
    
# else if this is running locally then GOOGLE_APPLICATION_CREDENTIALS should be defined
elif 'GOOGLE_APPLICATION_CREDENTIALS' in os.environ:
    with open(os.environ['GOOGLE_APPLICATION_CREDENTIALS'], 'r') as fp:
        credentials = json.load(fp)
    project_id = credentials['project_id']
else:
    raise Exception('Failed to determine project_id')

Gnosticize

Excellent and detailed, just a clarification your code works in cloud shell , also if i understand, if i dont set any environment id like GCP_Project is it not possible for me to get the project id of the cloud function where it is deployed? and I also tested GOOGLE_APPLICATION_CREDENTIALS (your elif part) my google_application_credentials is for project A, so if change the project to project B it still shows for project A only, as it is for that service account, so I change it to any other service account, is there a way? – Sepulchral

Using os.environ['GCP_PROJECT'] is deprecated and won't be available in subsequent runtimes. Users should use the metadata server instead. – Tolan

Answer 3

Google has released an official metadata library for Go, which can be found at cloud.google.com/go/compute/metadata. To get the project ID, you can simply use the ProjectID() function. For more details, please visit https://pkg.go.dev/cloud.google.com/go/compute/metadata#ProjectID.

Mummery