Amazon SageMaker: Invoke endpoint with file as multipart/form-data

Question

After setting up an endpoint for my model on Amazon SageMaker, I am trying to invoke it with a POST request which contains a file with a key image & content type as multipart/form-data.

My AWS CLI command is like this:

aws sagemaker-runtime invoke-endpoint --endpoint-name <endpoint-name> --body image=@/local/file/path/dummy.jpg --content-type multipart/form-data output.json --region us-east-1

which should be an equivalent of:

curl -X POST -F "image=@/local/file/path/dummy.jpg" http://<endpoint>

After running the aws command, the file is not transferred via the request, and my model is receiving the request without any file in it.

Can someone please tell me what should be the correct format of the aws command in order to achieve this?

Heisenberg

I think content-type might be tripping it off. Can you try changing content type as --content-type "application/x-image"? – Hepatic

Nope, that didn't help. – Heisenberg

Answer 1

The first problem is that you're using 'http' for your CURL request. Virtually all AWS services strictly use 'https' as their protocol, SageMaker included. https://docs.aws.amazon.com/general/latest/gr/rande.html. I'm going to assume this was a typo though.

You can check the verbose output of the AWS CLI by passing the '--debug' argument to your call. I re-ran a similar experiment with my favorite duck.jpg image:

aws --debug sagemaker-runtime invoke-endpoint --endpoint-name MyEndpoint --body image=@/duck.jpg --content-type multipart/form-data  >(cat)

Looking at the output, I see:

2018-08-10 08:42:20,870 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=InvokeEndpoint) (verify_ssl=True) with params: {'body': 'image=@/duck.jpg', 'url': u'https://sagemaker.us-west-2.amazonaws.com/endpoints/MyEndpoint/invocations', 'headers': {u'Content-Type': 'multipart/form-data', 'User-Agent': 'aws-cli/1.15.14 Python/2.7.10 Darwin/16.7.0 botocore/1.10.14'}, 'context': {'auth_type': None, 'client_region': 'us-west-2', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x109a58ed0>}, 'query_string': {}, 'url_path': u'/endpoints/MyEndpoint/invocations', 'method': u'POST'}

It looks like the AWS CLI is using the string literal '@/duck.jpg', not the file contents.

Trying again with curl and the "--verbose" flag:

curl --verbose -X POST -F "image=@/duck.jpg" https://sagemaker.us-west-2.amazonaws.com/endpoints/MyEndpoint/invocations

I see the following:

Content-Length: 63097

Much better. The '@' operator is a CURL specific feature. The AWS CLI does have a way to pass files though:

--body fileb:///duck.jpg

There is also a 'file' for non-binary files such as JSON. Unfortunately you cannot have the prefix. That is, you cannot say:

 --body image=fileb:///duck.jpg

You can prepend the string 'image=' to your file with a command such as the following. (You'll probably need to be more clever if your images are really big; this is really inefficient.)

 echo -e "image=$(cat /duck.jpg)" > duck_with_prefix

Your final command would then be:

 aws sagemaker-runtime invoke-endpoint --endpoint-name MyEndpoint --body fileb:///duck_with_prefix --content-type multipart/form-data  >(cat)

Another note: Using raw curl with AWS services is extremely difficult due to the AWS Auth signing requirements - https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html

It can be done, but you'll likely be more productive by using the AWS CLI or a pre-existing tool such as Postman - https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-use-postman-to-call-api.html

Evangelical

Figured all of these out and this was my next question. So basically, I want to send multiple images as file objects and get classifications from the model (which I can achieve by the CURL command, but I am not able to find a way to do the same using AWS CLI) – Heisenberg

@DipankarGhosh if you managed to do it, do not hesitate to post an answer here so that people who want to do the same thing, like me, can benefit from it, and can award you upvotes :) – Fernyak

I was struggling to invoke an endpoint with csv inputs then I saw your example with fileb:// and that extra 'b' fixed my issue so that I was posting a blob rather than plain text. Thank you! – Serpens

Answer 2

For anyone landing here and is OK to do it with python, here is what I found.

Attribution: https://betatim.github.io/posts/python-create-multipart-formdata/

Send Form Data with a file to sagemaker endpoint with python

Use urllib3.encode_multipart_formdata

Example:

import boto3
import sagemaker
from sagemaker.predictor import Predictor
from urllib3 import encode_multipart_formdata

ENDPOINT_NAME = "example-endpoint"
IMAGE_PATH = "examples/image.jpg"
KEY = "image"
FILENAME = "image.jpg"

session = boto3.Session()
sagemaker_session = sagemaker.Session(boto_session=session)
predictor = Predictor(ENDPOINT_NAME, sagemaker_session)
data = {
  KEY: (FILENAME, open(IMAGE_PATH, "rb").read(), "image/jpg")
}

body, header = encode_multipart_formdata(data)
inference_response = predictor.predict(body, initial_args={"ContentType": header})

print(inference_response)

Procto

Thank you very much. Wish I could upvote this more than once. For anyone using boto3 client, such as my_client = boto3.client('sagemaker-runtime', region_name=your_region_name), once you prepare your data, body and header as described in the answer above, you can also do my_client.invoke_endpoint(EndpointName=your_endpoint_name, ContentType=header, Body=body)['Body'].read().decode() to get your output. – Hydrastis