How to set the request timeout in google ml api python client?

Question

I'm running online predictions on google cloud machine learning API using the google api python client and a model hosted for me at google cloud. When I predict sending one image, the server, including all traffic, is taking about 40 seconds. When I send two images, after some time, I receive the message:

timeout: The read operation timed out

I would like to set the timeout to other value, but I didn't find how.

This is my code:

import base64
import io
import time
from PIL import Image    

from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery

SCOPES = ['https://www.googleapis.com/auth/cloud-platform']
SERVICE_ACCOUNT_FILE = 'mycredentialsfile.json'

credentials = ServiceAccountCredentials.from_json_keyfile_name(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES)

ml = discovery.build('ml', 'v1', credentials=credentials)

projectID = 'projects/{}'.format('projectID') + '/models/{}'.format('modelID')

width = 640
height = 480

instances = []

for image in ["image5.jpg", "image6.jpg"]:
    img = Image.open(image)
    img = img.resize((width, height), Image.ANTIALIAS)
    output_str = io.BytesIO()
    img.save(output_str, "JPEG")
    instance = {"b64": base64.b64encode(output_str.getvalue()).decode("utf-8") }
    output_str.close()
    instances.append(instance)  

input_json = {"instances": instances }

request = ml.projects().predict(body=input_json, name=projectID)

print("Starting prediction")
start_time = time.time()
response = request.execute()

print("%s seconds" % (time.time() - start_time))

Marietta

Answer 1

[update] please see @Sylver11 and @Shohei's answers for the explanation of the full implication of this solution.

It took me a while to find a simple resolution. You only need to add the following to the code

import socket
timeout_in_sec = 60*3 # 3 minutes timeout limit
socket.setdefaulttimeout(timeout_in_sec)

# then you could create your ML service object as usually, and it will have the extended timeout limit.
ml_service = discovery.build('ml', 'v1')

#however, this is a hacky solution because this a low level setting could also impact other http clients. so, please set it back
socket.setdefaulttimeout(None)

Pomiferous

Answer 2

I found a way researching samples from google api python client on github and trying same changes.

Using the httplib2 to authenticate you can set the timeout.

Following the modified code:

import base64
import io
import time
from PIL import Image

# Need: pip install google-api-python-client

import httplib2
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery

SCOPES = ['https://www.googleapis.com/auth/cloud-platform']
# API & Services -> Credentials -> Create Credential -> service account key
SERVICE_ACCOUNT_FILE = 'mycredentialsfile.json'

credentials = ServiceAccountCredentials.from_json_keyfile_name(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES)

http = httplib2.Http(timeout=200)
http = credentials.authorize(http)

ml = discovery.build('ml', 'v1', http=http)

projectID = 'projects/{}'.format('projectID ') + '/models/{}'.format('modelID')

width = 640
height = 480

instances = []

for image in ["image5.jpg", "image6.jpg"]:
    img = Image.open(image)
    img = img.resize((width, height), Image.ANTIALIAS)
    output_str = io.BytesIO()
    img.save(output_str, "JPEG")
    instance = {"b64": base64.b64encode(output_str.getvalue()).decode("utf-8") }
    output_str.close()
    instances.append(instance)  

input_json = {"instances": instances }

request = ml.projects().predict(body=input_json, name=projectID)

print("Starting prediction")
start_time = time.time()
response = request.execute()

print("%s seconds" % (time.time() - start_time))

I think with a few modifications you can use this to set timeout to almost any google cloud API in python client.

I hope this helps.

Marietta

Unfortunately ran into: googleapiclient.errors.HttpError: <HttpError 401 when requesting https://ml.googleapis.com/v1/projects/lobe-168020/models/lobe_86797be8_5849_404d_a45d_6bbe8da4d4e5:predict?alt=json returned "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.">, will keep looking around – Francenefrances

Hi @Francenefrances ! Looks like you have a problem with your credentials, do you create your own "mycredentialsfile.json file" in the google cloud platform console? If no, probably this is the problem reason. If yes, the problem could be the type of your credentials file. – Marietta

Hi, there is a easier way to handle this. you could just use socket.setdefaulttimeout(timeout_in_sec). Other answers below in this thread give more details. – Pomiferous

Answer 3

You have already solved the problem, but I found the other way to do this.

import socket    
socket.setdefaulttimeout(150)

If call discovery.build without http, http client is instantiated by build_http in build method.

https://googleapis.github.io/google-api-python-client/docs/epy/googleapiclient.http-pysrc.html#build_http

As you can see here, build_http create a http client instance with timeout if it is set before creating http client.

So all you have to do is setting this value by socket.setdefaulttimeout :)

Epiphora

yes. I agree with Shohei, It took me a while to find this simple and elegant resolution. All you need is – Pomiferous

Answer 4

I don't agree with the accepted answer as this can lead to limitations when using the api. I found the corresponding code here: https://github.com/googleapis/google-api-python-client/blob/main/googleapiclient/http.py#L1933-L1962

This is what happens inside:

def build_http():
"""Builds httplib2.Http object

Returns:
A httplib2.Http object, which is used to make http requests, and which has timeout set by default.
To override default timeout call

  socket.setdefaulttimeout(timeout_in_sec)

before interacting with this method.
"""
if socket.getdefaulttimeout() is not None:
    http_timeout = socket.getdefaulttimeout()
else:
    http_timeout = DEFAULT_HTTP_TIMEOUT_SEC
http = httplib2.Http(timeout=http_timeout)
# 308's are used by several Google APIs (Drive, YouTube)
# for Resumable Uploads rather than Permanent Redirects.
# This asks httplib2 to exclude 308s from the status codes
# it treats as redirects
try:
    http.redirect_codes = http.redirect_codes - {308}
except AttributeError:
    # Apache Beam tests depend on this library and cannot
    # currently upgrade their httplib2 version
    # http.redirect_codes does not exist in previous versions
    # of httplib2, so pass
    pass

return http

As you can see there are further configurations added to the http instance in the try block. So I would recommend @Shohei Miyashita approach with the socket.

That being said I do not think that this is a good solution either because this is a low level setting meaning that it could apply to other http clients. What I would recommend is setting it back to its default value after the API client has been instantiated:

import socket    
socket.setdefaulttimeout(150)
# instantiate API client here    
sock = socket.socket()
sock.settimeout(None)

Infra