Python3 Asyncio shared resources between concurrent tasks

Question

I've got a network application written in Python3.5 which takes advantage of pythons Asyncio which concurrently handles each incoming connection.

On every concurrent connection, I want to store the connected clients data in a list. I'm worried that if two clients connect at the same time (which is a possibility) then both tasks will attempt to write to the list at the same time, which will surely raise an issue. How would I solve this?

Spirituel

if you're doing HTTP you should use aiohttp, which (AFAIK) takes care of all of that for you. – Adalbertoadalheid

Answer 1

asyncio does context switching only on yield points (await expressions), thus two parallel tasks are not executed at the same time.

But if race conditions are still possible (it depends on concrete code structure) you may use asyncio synchronization primitives and queues.

Crawl

Each client connection is handled by an individual task. Within each task, when the client is being handled, data is appended to a global list. task = asyncio.Task(self._handle_client(client_reader, client_writer)) self.clients[task] = (client_reader, client_writer) and within _handle_client the list is being appended to. Will this cause an issue if two clients connect at the exact same time? – Spirituel

No, it's pretty safe. – Crawl

Answer 2

There is lots of info that is missing in your question.

• Is your app threaded? If yes, then you have to wrap your list in a threading.Lock.
• Do you switch context (e.g. use await) between writes (to the list) in the request handler? If yes, then you have to wrap your list in a asyncio.Lock.
• Do you do multiprocessing? If yes then you have to use multiprocessing.Lock
• Is your app divided onto multiple machines? Then you have to use some external shared database (e.g. Redis).

If answers to all of those questions is no then you don't have to do anything since single-threaded async app cannot update shared resource parallely.

Root