Django - post InMemoryUploadedFile to external REST api

Question

In the Django Rest Framework I would like to post a file, received as an InMemoryUploadedFile, to a different server as soon as it is received.

It sounds simple, but the request.post() function does not seem to properly send over such a file :

def post(self, request, *args, **kwargs):
    data = request.data
    print(data)
    # <QueryDict: {'file': [<InMemoryUploadedFile: myfile.pdf (application/pdf)>]}>
    endpoint = OTHER_API_URL + "/endpoint"
    r = requests.post(endpoint, files=data)

My other server receives the request (through flask) with the name of the file, but not the content:

@app.route("/endpoint", methods=["POST"])
def endpoint():
    if flask.request.method == "POST":
        # I removed the many checks to simplify the code
        file = flask.request.files['file']
        path = os.path.join(UPLOAD_FOLDER, file.filename)
        file.save(path)        

        print(file) #<FileStorage: u'file.pdf' (None)>
        print(os.path.getsize(path)) #0

        return [{"response":"ok"}]

When posting a file directly to that api in form-data with postman, It works as expected:

        print(file) # <FileStorage: u'file.pdf' ('application/pdf')>
        print(os.path.getsize(path)) #8541

Any help on how to fix this, i.e. transform the InMemoryUploadedFile type in something a normal REST api can understand? Or maybe just adding the right headers?

Lymanlymann

Can you include the relevant Flask view, to see how the files are handled on the receiving end? – Trichromat

it contains self.request.FILES – Welterweight

@Trichromat I edited the question with the details. – Lymanlymann

Are there any proxies between the two servers, something that could alter the request to the Flask server? – Trichromat

No, the two servers are two separate docker containers – Lymanlymann

The only (very inefficient) way I manage to make it work is: 1. save the file on the drive of the first server 2. open it again with open(path, 'rb') 3. make the post request to the second server 4. delete the file from my first server – Lymanlymann

Answer 1

I had to figure this issue out passing an uploaded file from a Django front end website to a Django backend API in Python 3. The InMemoryUploadedFile's actual file data can be accessed via the object's .file property's .getvalue() method.

        path="path/to/api"
        in_memory_uploaded_file = request.FILES['my_file']
        io_file = in_memory_uploaded_file.file
        file_value = io_file.getvalue()
        files = {'my_file': file_value}
        make_http_request(path, files=files)

and can be shortened

        file = request.FILES['my_file'].file.getvalue()
        files = {'my_file': file}

Before this, trying to send InMemoryUploadFile objects, the file property, or the result of the read() method all proved to send a blank/empty file by the time it got to the API.

Limburg

i tried the following code. but doesn't work for me. – Sidereal

view.py- kyc_file = request.FILES['kyc_doc'].file.getvalue() files = {"kyc_doc":kyc_file} resp = accService.publishKY(data,files) -------- Service.py -- def publishKY(self,data,files): curl = CurlWrapper() response_kcloud = curl.post(endpoint,data,{ "api-key":acc.api_key,"Content-Type":"multipart/form-data"},files) ------ class CurlWrapper(): def post(self,url,data,headers,ufile=None): if ufile is None: return requests.post(url, data=json.dumps(data), headers=headers) else: return requests.post(url, data=data, headers=curlheaders,files=ufile) – Sidereal

Answer 2

I had the same problem and the same case. My working solution

    headers = {
        "Host": API_HOST,
        "cache-control": "no-cache",
    }

    try:
        data = json_request = request.POST['json_request'].strip()
        data = json.loads(data) # important!
    except:
        raise Http404

    try:
        in_memory_uploaded_file = request.FILES['file'].file.getvalue() 
        files = {'photo': in_memory_uploaded_file} # important!
    except:
        files = {}

    if USE_HTTPS:
        API_HOST = f'https://{API_HOST}'
    else:
        API_HOST = f'http://{API_HOST}'

    if authorization_key and len(authorization_key) > 0:
        response = requests.post(f'{API_HOST}/api/json/?authorization_key={authorization_key}', headers=headers, data=data, files=files)
    else:    
        response = requests.post(f'{API_HOST}/api/json/', headers=headers, data=data)

    json_response = json.dumps(response.json())

Mims

Hi! This gives me ValueError: I/O operation on closed file. – Mozzetta

Answer 3

handler = flask.request.files['file']

files =  [('<destination_api_endpoint_argument>', (handler.name, handler.read(), handler.content_type)]


resp = requests.post(url, data = data, headers=curlheaders, files = files)

Christopherchristopherso

It would be helpful to add some explanations here. This seems like a useful answer, but explanations would help readers understand it much better. – Glorification