How to post Multipart Form Data through python aiohttp ClientSession

Question

I am trying to asynchronously send some Multipart-Encoded Form Data as a post request, mainly a file and two other fields.

Before trying to use asyncio I was doing the process synchronously with requests-toolbelt MultipartEncoder (https://github.com/requests/toolbelt) which worked great for normal requests, but did not work when using aiohttp for async. aiohttp provides 2 multipart classes, a FormData() class and a MultipartWriter() class, neither of which have given me much success.

After some testing, it seems like the difference is that when I use the toolbelt MultipartEncoder() the request sends the data in the form section of the post request as it should. However, when using aiohttp the request is put into the body section of the request. Not sure why they are acting differently

def multipartencode() -> ClientResponse():
        # Using MultipartEncoder
        m = MultipartEncoder(
            fields={'type': type_str,
                    'metadata': json.dumps(metadata),
                    'file': (filename, file, 'application/json')}
        )

        # Using FormData
        data = FormData()
        data.add_field('file', file, filename=filename,
                       content_type='multipart/form-data')
        data.add_field('type', type_str, content_type='multipart/form-data')
        data.add_field('metadata', json.dumps(metadata),
                       content_type='multipart/form-data')

        # Using MultipartWriter
        with MultipartWriter('multipart/form-data') as mpwriter:
            part = mpwriter.append(
                file, {'CONTENT-TYPE': 'multipart/form-data'})
            part.set_content_disposition('form-data')
            part = mpwriter.append_form([('type', type_str)])
            part.set_content_disposition('form-data')
            part = mpwriter.append_form([('metadata', json.dumps(metadata))])
            part.set_content_disposition('form-data')


        # send request with ClientSession()
        resp = await session.post(url=url, data=data, headers=headers)
        return resp

How can I properly format/build the multipart-encoded request to get it to send using aiohttp?

Arteriovenous

This issue posted on aiohttp might be helpful github.com/aio-libs/aiohttp/issues/3571, specifically this comment: github.com/aio-libs/aiohttp/issues/3571#issuecomment-456509079 – Mainspring

did you find solution? – Freedom

Answer 1

I was struggling with this for hours. My particular case was to send an email with file attachment to mailgun. Same thing that was addressed by the comment above. Please find below the working code:

import asyncio

import aiohttp


async def send():
    url = "<<mailgun_url>>"
    api_key = "<<mailgun_api_key>>"

    mail_gun_data = {
        "from": "<<from>>",
        "to": "<<to>>",
        "subject": "Subject",
        "text": "Testing Mailgun",
        "attachment": open("<<file_path>>", "rb")
    }

    async with aiohttp.ClientSession() as session:
        with aiohttp.MultipartWriter("form-data") as mp:
            for key, value in mail_gun_data.items():
                part = mp.append(value)
                part.set_content_disposition('form-data', name=key)
            resp = await session.post(
                url,
                auth=aiohttp.BasicAuth("api", api_key),
                data=mp,
            )


if __name__ == '__main__':
    asyncio.run(
        send()
    )

I hope this is helpful and will save someone's time.

Hostetler

Answer 2

I was struggling with the same issue and found there was no easy solution. What happened is that the FormData class adds a few other dispositions include filename=${file_name}link1, name=filelink2 in here. There is a potential other one here. When payload is appended to the MultiWriter, the 'content-length' is also added to the header here. When I run the code in debug mode, I also found the "content-type" of the payload is somehow resolved to the correct type based on the file.

I can't imagine how much other things are done by the package inherently. I would stay away from writting my own MultiWriter if possible..

Philemol