How can I send an HTTP request from my FastAPI app to another site (API)?

Question

I am trying to send 100 requests at a time to a server http://httpbin.org/uuid using the following code snippet

from fastapi import FastAPI
from time import sleep
from time import time
import requests
import asyncio

app = FastAPI()

URL= "http://httpbin.org/uuid"


# @app.get("/")
async def main():
    r = requests.get(URL)
    # print(r.text)
    
    return r.text

async def task():
    tasks = [main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main()]
    # print(tasks)
    # input("stop")
    result = await asyncio.gather(*tasks)
    print (result)

@app.get('/')
def f():
    start = time()
    asyncio.run(task())
    print("time: ",time()-start)

I am using FastAPI with Asyncio to achieve the lowest time possible around 3 seconds or less but using the above method I am getting an overall time of 66 seconds that is more than a minute. I also want to keep the main function for additional operations on r.text. I understand that to achieve such low time, concurrency is required but I am not sure what mistake I'm doing here.

Penthea

even with curl i'm getting 0.111 ms for 100 requests time for _ in {1..100}; do curl http://httpbin.org/uuid & done this is pretty weird. – Plywood

it's because each curl is splitting into a different process but the same thing is not working in my code - imgur.com/a/pmh7qLb time for _ in {1..100}; do curl http://httpbin.org/uuid & done && ps aux | grep curl – Penthea

IKR i was talking about this, asyncio should behave exactly like curl, i tried on my machine with different approach that didn't worked out too, then i thinked about uvicorn is just an another event loop maybe it's the issue, after that i ran it normally, also it took so long tho.. – Plywood

Also i'm using this pattern for asyncio, which looks pretty solid for me, i created a gist you might want to check it out – Plywood

Yep, Even I find this right. – Penthea

My machine died thrice even for range(10). – Penthea

So, a combination of asyncio and aiohttp would help in the case of URL request scenarios? – Penthea

You could also try python-httpx.org . Create an AsyncClient with the base url, then use the client (equivalent of a request session) to perform 100 calls to the url – Phila

Thanks, I will look at this. – Penthea

@johnmich hey, i needed to use my gist again, then I found I used main() inside main, it should be working as expected now, sorry for killing your machine :( – Plywood

Answer 1

requests is a synchronous library. You need to use an asyncio-based library to make requests asynchronously.

httpx

httpx is typically used in FastAPI applications to request external services. It provides synchronous and asynchronous clients which can be used in def and async def path operations appropriately. It is also recommended for asynchronous tests of application. I would advice using it by default.

from fastapi import FastAPI
from time import time
import httpx
import asyncio

app = FastAPI()

URL = "http://httpbin.org/uuid"


async def request(client):
    response = await client.get(URL)
    return response.text


async def task():
    async with httpx.AsyncClient() as client:
        tasks = [request(client) for i in range(100)]
        result = await asyncio.gather(*tasks)
        print(result)


@app.get('/')
async def f():
    start = time()
    await task()
    print("time: ", time() - start)

Output

['{\n  "uuid": "65c454bf-9b12-4ba8-98e1-de636bffeed3"\n}\n', '{\n  "uuid": "03a48e56-2a44-48e3-bd43-a0b605bef359"\n}\n',...
time:  0.5911855697631836

aiohttp

aiohttp can also be used in FastAPI applications, if you prefer one.

from fastapi import FastAPI
from time import time
import aiohttp
import asyncio

app = FastAPI()

URL = "http://httpbin.org/uuid"


async def request(session):
    async with session.get(URL) as response:
        return await response.text()


async def task():
    async with aiohttp.ClientSession() as session:
        tasks = [request(session) for i in range(100)]
        result = await asyncio.gather(*tasks)
        print(result)


@app.get('/')
async def f():
    start = time()
    await task()
    print("time: ", time() - start)

---

If you want to limit the number of requests executing in parallel, you can use asyncio.semaphore like so:

MAX_IN_PARALLEL = 10
limit_sem = asyncio.Semaphore(MAX_IN_PARALLEL)


async def request(client):
    async with limit_sem:
        response = await client.get(URL)
        return response.text

Publea

You might want to create a single session in task and pass it to main(). That will allow aiohttp to be more efficient by e.g. reusing TCP connections. – Waterspout

"It is a dependency for FastAPI." - It's not. – Labroid

This is a dependency only for fastapi[test], so I removed the phrase to avoid confusing people. – Publea

@any reason to use httpx? versus aiohttp? I ask because httpx is considered to be in beta version, so in production, you want the "more" stable one, thx – Herrington

@AlexJolig the page you've linked to is dated 2021-04-17. This answer is dated 2020-09-14. The page's title is identical to this question. Perhaps you should direct your remark to the maintainer of that site, and retract your accusation towards this answer's author. – Ineffective

"[B]ut do so if you really need it" makes it sound like aiohttp, a well-known, established library, might cause significant problems in a FastAPI application, but your answer doesn't explain what those might be. Could you please edit your answer to explain? – Jujutsu

@Julia, this will not cause any special problems, so I slightly corrected the answer – Publea

Great! Thank you, Alex! – Jujutsu

Answer 2

@Alex Noname has made a good point of using asynchronous request library. If you want to make code faster I would suggest using asyncio.Queue as an alternate. In this example I spun up 100 producers and 100 consumers. you can limit the maximum number of messages in the queue like, then producer waits until there is space for new message

asyncio.Queue(maxsize=100)

also I have made use of AsyncClient from httpx.

If you want to know more about queues I would suggest this article https://realpython.com/async-io-python/

from time import time
from typing import List

from fastapi import FastAPI
from httpx import AsyncClient

app = FastAPI()

URL = "http://httpbin.org/uuid"
client = AsyncClient()


async def main():
    r = await client.get(URL)
    return r.text


async def producer(queue: asyncio.Queue):
    await queue.put(main)


async def consumer(queue: asyncio.Queue, resp: List):
    # await queue.get() == main  -> without arguments
    resp.append(await (await queue.get())())


async def task():
    q = asyncio.Queue(maxsize=100)
    response = []
    consumers = []
    producers = []
    [consumers.append(consumer(q, response)) for c in range(100)]
    [producers.append(producer(q)) for p in range(100)]

    await asyncio.gather(*producers)
    await asyncio.gather(*consumers)
    print(response)


@app.get('/')
def f():
    start = time()
    asyncio.run(task())
    print("time: ", time() - start)


if __name__ == '__main__':
    f()

output

['{\n  "uuid": "a7713d07-ea5d-40d3-95b4-6673f3c50a8b"\n}\n', '{\n  "uuid": "c93f8b89-2c44-40fa-9e5f-736e22ad5f23"\n}\n', '{\n  "uuid": "cbb4ad76-7790-45ae-87f1-e425eddc8021"\n}\n', '{\n  "uuid": "4c1d81c0-ae7d-401a-99df-e98af3651335"\n}\n', '{\n  "uuid": "c5f70738-fbba-4cf9-8fdf-29f8b4eabe63"\n}\n', '{\n  "uuid": "d016b852-4312-4502-a336-a6a110237d1d"\n}\n', '{\n  "uuid": "22d8b00b-4266-4236-b5a3-ed5d7c5be416"\n}\n', '{\n  "uuid": "cd54fdbb-7de9-4df3-90cc-e6b108d5fdf8"\n}\n', '{\n  "uuid": "757f0a26-7896-4a04-bea2-60c66a38b05b"\n}\n', '{\n  "uuid": "72eb6584-21f4-449b-b6bd-d0f88666126f"\n}\n', '{\n  "uuid": "b3deadf5-5b79-491b-829c-0404c306cb68"\n}\n', '{\n  "uuid": "789e7422-493d-49d2-9585-e5ca34b7cf36"\n}\n', '{\n  "uuid": "48d29a82-ff7c-41f5-8af2-42784326a31f"\n}\n', '{\n  "uuid": "84b2d67c-331c-4037-b6e4-c299d93c1899"\n}\n', '{\n  "uuid": "386e79f9-073a-4f27-961c-7befcdf95cd4"\n}\n', '{\n  "uuid": "8dfdb5e4-dd69-4043-b174-48ec8505f36f"\n}\n', '{\n  "uuid": "633e634b-b107-42bb-a7d3-c6bbfff089a0"\n}\n', '{\n  "uuid": "962d665f-8663-4be7-a3c6-9426ba500bf4"\n}\n', '{\n  "uuid": "320fb858-a751-4c34-9cdb-ddd2f4e28efa"\n}\n', '{\n  "uuid": "46a75693-5255-4ac7-8d7a-54910b4d6f68"\n}\n', '{\n  "uuid": "5323734b-7ff9-455e-ba5a-66383e6b9a1f"\n}\n', '{\n  "uuid": "622a579f-35b6-4e4b-9dba-a8a69c2049c8"\n}\n', '{\n  "uuid": "593d5e82-cef3-4be0-99ab-e3034855d7a1"\n}\n', '{\n  "uuid": "80f139df-2a27-40c1-8329-e4faa035c45c"\n}\n', '{\n  "uuid": "a97e084c-4d30-4c7b-a96e-89ed00dcfe2a"\n}\n', '{\n  "uuid": "360d49eb-7222-4064-81c2-6eba2d43a9a5"\n}\n', '{\n  "uuid": "a81b6eab-a646-4e58-b986-96a90baa52aa"\n}\n', '{\n  "uuid": "0160337e-b400-41d6-ae89-aa46c5131f40"\n}\n', '{\n  "uuid": "e600722f-8c15-4959-948b-4c4e5296feb2"\n}\n', '{\n  "uuid": "f15403e4-3674-43b2-a0c9-649fd828ba7e"\n}\n', '{\n  "uuid": "36bf139c-cc18-45a8-bc55-e7f90ce290b5"\n}\n', '{\n  "uuid": "b2368a3c-d86b-4fcd-a0d3-bf7f8f657a83"\n}\n', '{\n  "uuid": "d9f16c36-3572-4c70-8a41-3d4e279d76bf"\n}\n', '{\n  "uuid": "796087cc-a202-40dd-9921-14802a73323d"\n}\n', '{\n  "uuid": "089fa0d7-4c48-4daa-a80d-cb5ebd37dfb7"\n}\n', '{\n  "uuid": "e5582bc7-0f8a-4da7-b640-79a0d812154d"\n}\n', '{\n  "uuid": "bac0640b-0d0b-4bf2-a3c1-36bdda7cce03"\n}\n', '{\n  "uuid": "b4353004-02b2-4846-8692-33dd77ad1d3f"\n}\n', '{\n  "uuid": "1b34a744-d0ea-4acf-8bda-33743800d86a"\n}\n', '{\n  "uuid": "4d9dd269-6ee2-4356-9bc4-ddf188445320"\n}\n', '{\n  "uuid": "a1f380df-0c0d-4aee-bbb7-c3e99fbfe54f"\n}\n', '{\n  "uuid": "7cb762eb-1a42-433d-97ea-aa9de4504e35"\n}\n', '{\n  "uuid": "981c40e2-64bf-4746-8103-9430bda2a5ca"\n}\n', '{\n  "uuid": "22b778eb-82d1-48b9-9874-5ebb80ddb8b1"\n}\n', '{\n  "uuid": "e7a9e0e8-7964-400c-aafe-9c36b9b7e1a0"\n}\n', '{\n  "uuid": "21a59b91-2732-4bb6-a47e-84008a03c20c"\n}\n', '{\n  "uuid": "a78eeb39-5ecb-4509-87c2-b4a2529e3536"\n}\n', '{\n  "uuid": "4a332579-ce03-4f69-9db5-78da9196d6b2"\n}\n', '{\n  "uuid": "55fbc34f-4eb3-4356-98e3-1df38054a4b2"\n}\n', '{\n  "uuid": "257ac454-09c2-4fd4-bdb3-303495360fa2"\n}\n', '{\n  "uuid": "7505cc0d-01b3-47f8-91d4-3e54d0f387de"\n}\n', '{\n  "uuid": "0fd67af2-622e-4688-b3c8-f64e20f1f3ec"\n}\n', '{\n  "uuid": "07653ccf-f408-4807-8ff5-e6098d657451"\n}\n', '{\n  "uuid": "b9d0ff18-fd67-4afa-adbe-ebcb53380804"\n}\n', '{\n  "uuid": "70d4d53b-2f06-41be-bb38-47f010cfa40f"\n}\n', '{\n  "uuid": "a6d49873-e749-4578-ae9c-e6c6f473535d"\n}\n', '{\n  "uuid": "e67efee5-76ad-4812-bb97-016ef9ff87e8"\n}\n', '{\n  "uuid": "67886926-b2d9-44fb-b836-26b81c53e5fb"\n}\n', '{\n  "uuid": "dcbd4ff8-e3cd-4e03-b12d-5fb3834b0e00"\n}\n', '{\n  "uuid": "65c2eaee-5fa2-4b58-a1c3-adeb04d92c71"\n}\n', '{\n  "uuid": "2cee4ec9-952e-45c5-91b7-f4f5848c3455"\n}\n', '{\n  "uuid": "8e94bf1c-ee5a-483a-a962-d0b9aea48c95"\n}\n', '{\n  "uuid": "c1fe17bc-bedf-4c4c-952d-a5921f693d9f"\n}\n', '{\n  "uuid": "221456fd-48ca-4826-a8b5-5fa0b23db6e4"\n}\n', '{\n  "uuid": "62fda759-b382-44e4-ad7d-d19a952fc1c7"\n}\n', '{\n  "uuid": "73faeb91-215e-4e49-8f11-11b98e499cc7"\n}\n', '{\n  "uuid": "f3279c45-ebcc-4079-b823-3efe825c7cf8"\n}\n', '{\n  "uuid": "b892672b-4510-44f4-b61e-9cccaa52421e"\n}\n', '{\n  "uuid": "8926979d-71a7-4171-9389-ddafff89e229"\n}\n', '{\n  "uuid": "d97cef59-4862-42ca-b0f2-261f98fd4b6f"\n}\n', '{\n  "uuid": "3362ff93-89e4-4889-a2f2-2e03771e86ce"\n}\n', '{\n  "uuid": "9f525251-4fe4-4a9c-97b5-2f01d2b37aaf"\n}\n', '{\n  "uuid": "036959d4-3179-40f9-bbf3-32274f2cede2"\n}\n', '{\n  "uuid": "157f8c22-6214-4e27-ab5d-08d39f96d1d3"\n}\n', '{\n  "uuid": "e4bfbf62-7c33-4fd7-a231-47f5ce398041"\n}\n', '{\n  "uuid": "a41512c1-3346-4457-a379-64d690ffc2ea"\n}\n', '{\n  "uuid": "7bb07cfb-294b-44fa-a8dc-6d283c54409f"\n}\n', '{\n  "uuid": "f2297d22-a2d0-47ff-8d65-24c6fe7877a7"\n}\n', '{\n  "uuid": "645e255b-4c93-4c8f-9ff2-43da293db660"\n}\n', '{\n  "uuid": "9190e370-dfa9-47a6-8cef-8df7ab762433"\n}\n', '{\n  "uuid": "83216551-9f1b-48b2-8cd6-fd125a7ce965"\n}\n', '{\n  "uuid": "aaddb98c-879b-472d-aa39-1a684ef7d179"\n}\n', '{\n  "uuid": "4bd7e2fd-1453-4433-aa9f-bc29d82f5b9d"\n}\n', '{\n  "uuid": "b02d65e8-2063-4060-96af-088ec497fc10"\n}\n', '{\n  "uuid": "e10e3dd2-83c5-4595-afe4-4145bce79193"\n}\n', '{\n  "uuid": "8cb62784-1b5d-4dcc-8342-02ad7d417ca9"\n}\n', '{\n  "uuid": "13ef1509-4f69-4426-ac42-cb29a2d0f094"\n}\n', '{\n  "uuid": "4d4571d5-69bb-4625-b246-b5eef50aa10d"\n}\n', '{\n  "uuid": "75e7a2ca-bfa8-43b9-b33a-f3f927453579"\n}\n', '{\n  "uuid": "0a8cc8ff-2039-4873-9e38-afad3e10d726"\n}\n', '{\n  "uuid": "189ae75b-4879-4897-9725-f9be17e49844"\n}\n', '{\n  "uuid": "ba482468-f45f-4060-a0c1-3ef31bb283c8"\n}\n', '{\n  "uuid": "3809f1c7-2f11-487d-bf96-8abf64e08298"\n}\n', '{\n  "uuid": "da5ea88b-974d-4238-9654-ac56c657c8b4"\n}\n', '{\n  "uuid": "edc3de79-7cf4-42a3-a5f4-b754136a6fd3"\n}\n', '{\n  "uuid": "6f5ecd91-537c-4009-8435-6c31ce035d36"\n}\n', '{\n  "uuid": "4a33b29d-78ba-468f-8f30-a01b3d9e2a87"\n}\n', '{\n  "uuid": "a5a2ef2d-d4a2-48e1-8335-f8c1309328c4"\n}\n', '{\n  "uuid": "3d1679da-afdd-4f04-9c16-0aaea4c53d0c"\n}\n', '{\n  "uuid": "c4025845-0d4c-4549-acb8-1a249b33e644"\n}\n']
time:  1.0535461902618408

Parathyroid

Can you explain how is this code faster than in the other answer? In the example there are couple programming flaws one might want to avoid, like appending an item to an external list inside a comprehension list instead of using the comprehension list directly. Also the code will hang forever when q's maxsize is below 100, as the producers will eventually hang on queue.put() and so the gather(*producers) never completes. You'd have to schedule the two gathers as independent tasks, but that's not happening here. Downvoted. – Locoweed

what problem are you solving? – Samsun

Answer 3

you can use the asyncio.to_thread to covert sync requests to async if you wish to not use aiohttp or httpx

async def async_request(url,payload,**kwargs):   
    return await asyncio.to_thread(requests.post,url,payload)

async def main():
   await async_request(url,payload)

Lumpfish