Google's Vision Api protobuf response object to Python dictionary

Question

I'm working on a project in which I need to analyze an image using Google's Vision API and post the response to a Dynamodb table.

I have successfully implemented the Vision API, but not able to convert its response into Python Dictionary.

Here's what I have tried:

       if form.is_valid():
            obj = form
            obj.imageFile = form.cleaned_data['imageFile']
            obj.textFile = form.cleaned_data['textFile']
            obj.save()
            print(obj.imageFile)
            # Process the image using Google's vision API
            image_path = os.path.join(settings.MEDIA_ROOT, 'images/', obj.imageFile.name)
            print(image_path)
            image = vision_image_manager(image_path)
            text_path = os.path.join(settings.MEDIA_ROOT, 'texts/', obj.textFile.name)
            text = nlp_text_manager(text_path)
            # print(image)
            # print(text)
            results = {
                'imageResponse': image,
                'textResult': text
            }
            print(results.values())
            print(type(results))
            post_to_dynamo_db(image, text)

Here's the Vision api implementation:

def vision_image_manager(image_file):
    # Instantiates a client
    client = vision.ImageAnnotatorClient()
    file_name = str(image_file)
    with open(file_name, 'rb') as img_file:
        content = img_file.read()
    image = types.Image(content=content)
    response = client.label_detection(image=image)
    labels = response.label_annotations
    print('Labels:')
    for label in labels:
        print(label.description)
    return labels

And Here's the post_to_dynamo_db Function:

def post_to_dynamo_db(image, text):
session = boto3.Session(
    aws_access_key_id=settings.AWS_SERVER_PUBLIC_KEY,
    aws_secret_access_key=settings.AWS_SERVER_SECRET_KEY
)
client = session.resource('dynamodb')
table = client.Table('basetbl')
result_dict = {
    'image': image,
    'text': text
}
json_dict = dict_to_item(result_dict)
# item = dict_to_item(result_dict)
table.put_item(
    Item={
        'id': int(generate_pid()),
        'response_obj': json_dict
    }
)

Now, It doesn't return any error but the response_obj is not posted in Database table because it's not the correct form of the object, the problem here is the <class 'google.protobuf.pyext._message.RepeatedCompositeContainer'> type of response returns from Google's API.

Hydrolyze

Did you try using MessageToJson as in here? – Lustihood

Hi @GuillemXercavins, Yes I have tried it but it returns another error as: AttributeError:'google.protobuf.pyext._message.RepeatedCompositeCo' object has no attribute 'DESCRIPTOR' – Hydrolyze

Answer 1

The best way to get a proper python friendly response from Vision API is to use this API via the Google's Discovery service.

Here's how this will work for you:

def vision_image_manager(image_file):
    # Instantiates a client
    service = discovery.build('vision', 'v1', credentials=credentials)
    # text.png is the image file.
    file_name = str(image_file)
    with open(file_name, 'rb') as image:
        image_content = base64.b64encode(image.read())
        service_request = service.images().annotate(body={
            'requests': [{
                'image': {
                    'content': image_content.decode('UTF-8')
                },
                'features': [{
                    'type': 'LABEL_DETECTION',
                }]
            }]
        })
    response = service_request.execute()
    print(response['responses'])
    res_dict = dict(response)
    return res_dict

Hydrolyze