Deploy Retrained inception model on Google cloud machine learning

Question

I manage to retrain my specific classification model using the generic inception model following this tutorial. I would like now to deploy it on the google cloud machine learning following this steps.

I already managed to export it as MetaGraph but I can't manage to get the proper inputs and outputs.

Using it locally, my entry point to the graph is DecodeJpeg/contents:0 which is fed with a jpeg image in binary format. The output are my predictions.

The code I use locally (which is working) is:

softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
predictions = sess.run(softmax_tensor,{'DecodeJpeg/contents:0': image_data})

Should the input tensor be DecodeJpeg? What would be the changes I need to make if I would like to have a base64 image as input ?

I defined the output as:

outputs = {'prediction':softmax_tensor.name}

Any help is highly appreciated.

Apis

Answer 1

In your example, the input tensor is 'DecodeJpeg/contents:0', so you would have something like:

inputs = {'image': 'DecodeJpeg/contents:0')
outputs = {'prediction': 'final_result:0')

(Be sure to follow all of the instructions for preparing a model).

The model directory you intend to export should have files such as:

gs://my_bucket/path/to/model/export.meta
gs://my_bucket/path/to/model/checkpoint*

When you deploy your model, be sure to set gs://my_bucket/path/to/model as the deployment_uri.

To send an image to the service, as you suggest, you will need to base64 encode the image bytes. The body of your request should look like the following (note the 'tag', 'b64', indicating the data is base-64 encoded):

{'instances': [{'b64': base64.b64encode(image)}]}

Johannejohannes

Hello, note sure I understand how my request shoul look like. – Apis

I use gcloud beta ml predict --model inception --instances=file.json, and in the json file I have {'instances':[{'b64':b64_image(long string))}]}. The api respond Input instances must be in JSON format – Apis

If you're using gcloud, then you want your file to contain simply [{"b64": "really_long_base_64_encoded_string"}]. In other words, (1) gcloud adds the 'instances' bit and (2) you have to do the base64 encoding yourself – Johannejohannes

@Johannejohannes - I am trying to achieve a similar thing, deploy inception with a retrained last layer on cloud ml. I tried this tutorial, however, the frozen graph seems to support eval of only 1 image at a time, not variable batch sizes. I am in a process of retraining a newer version of inception from here, hoping the shapes will be more suitable. Is there by any chance any guide you know of I missed which addresses this? – Jacobs

We hope to provide more information for serving inception models soon. – Johannejohannes

Thanks :) I made it work with a dirty trick for now, looking forward to do it the right way. – Jacobs

@RobertLacok Mind sharing the hack to get this working, im stuck on the same problem? – Lourielouse

@Lourielouse I allowed the graph to accept variable batch size, but then passed only the first item to the jpeg decoder, and returned a single result. Pretty sure there are now examples out there which do it properly though. – Jacobs

Answer 2

We've now released a tutorial on how to retrain the Inception model, including instructions for how to deploy the model on the CloudML service.

https://cloud.google.com/blog/big-data/2016/12/how-to-train-and-classify-images-using-google-cloud-machine-learning-and-cloud-dataflow

Johannejohannes

Thanks, this will be very helpful – Apis