Google cloud vision not accepting base64 encoded images python

Question

I'm having a problem with base64 encoded images sent to Google Cloud Vision. Funny thing is that if I send the image via URI, it works fine, so I suspect there is something wrong the way I'm encoding.

Here's the deal:

from google.cloud import vision
import base64
client = vision.ImageAnnotatorClient()
image_path ='8720911950_91828a2aeb_b.jpg'
with open(image_path, 'rb') as image:
    image_content = image.read()
    content = base64.b64encode(image_content)   
    response = client.annotate_image({'image': {'content': content}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})
    print(response)

The response I get always is:

error {
  code: 3
  message: "Bad image data."
}

If I try using URI instead:

response = client.annotate_image({'image': {'source': {'image_uri': 'https://farm8.staticflickr.com/7408/8720911950_91828a2aeb_b.jpg'}}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})

Response is ok...

label_annotations {
  mid: "/m/0168g6"
  description: "factory"
  score: 0.7942917943000793
}
label_annotations {
  mid: "/m/03rnh"
  description: "industry"
  score: 0.7761002779006958
}

I've followed the recommended way to encode from Google

Any idea what is wrong here?

Haemocyte

Base64 != 64-bit. These are very different things. – Alluvial

Try content = base64.b64encode(image_content).decode() – Glenn

@Glenn I get this "TypeError: '/9j/4AAQSkZJRgABAQEA8ADwAAD/4gJASUNDX1BST0ZJTEUAAQEAAAIwQURCRQIQAABtbnRyUkdCIFhZWiAHzwAGAAMAAAAAAAB has type str, but expected one of: bytes " – Haemocyte

Have you tried the other way of doing it as in this example, where encoding is obviously done automatically inside the types.Image() constructor. – Glenn

@Glenn I don't want to use what they call shortcuts, I want to use the annotator client so I select what type of analysis, and more important, send more than one image per request – Haemocyte

My guess is that base64-encoding is performed by vision.ImageAnnotatorClient.annotate_image() automatically. Their documentation on base64 encoding is only for the case when you create and send the HTTP request on your own. So try eliminating base64 encoding: content = image.read() – Glenn

@Glenn I used a very similar approach with Google Cloud Vision not v1 and it worked fine... – Haemocyte

@Glenn That solved the issue! I understood this wrong from docs googlecloudplatform.github.io/google-cloud-python/stable/vision/… like if no JSON is going to be used then pure binary representation can be used – Haemocyte

Answer 1

I don't have any experience with Google Cloud Vision, however after looking at their documentation and examples, my feeling is that the linked documentation page about base64 encoding of image data is for the case when you create and send the HTTP requests on your own, without using vision.ImageAnnotatorClient. The latter seems to encode the image data automatically, hence in your example double encoding is applied. Therefore I believe that you should remove the encoding step from your code:

from google.cloud import vision
import base64
client = vision.ImageAnnotatorClient()
image_path ='8720911950_91828a2aeb_b.jpg'
with open(image_path, 'rb') as image:
    content = image.read()
    response = client.annotate_image({'image': {'content': content}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})
    print(response)

Glenn

correct! you got it right :) More info here, for the curious minds googlecloudplatform.github.io/google-cloud-python/stable/vision/… – Haemocyte

@Haemocyte It's a bit late but I'm trying to do the same thing and I was looking how to do it. The link that you gave is broken. Can you refresh it or share how can I encode the image to be used in Vision API? – Simper

@Simper let me know if this helps gist.github.com/internetmosquito/… – Haemocyte

@Haemocyte Thanks. – Simper

AttributeError: module 'google.cloud.vision' has no attribute 'enums' how can i get rid of this error? – Leucocytosis

Answer 2

Well, if you still want to use base64 encoded image data, you will have to convert it into byte array using module before sending request to annotate image. This base64 to bytearray should be used when creating API or when you are receiving input in the form of encoded data without actual path/url. Otherwise, use as it is by providing path or url as pointed out by @Leon.

import binascii
content = binascii.a2b_base64(base64_encoded_image_data)

pass this content as value for content argument in annotate_image method. Then, you will get a correct response.

Bareilly