RuntimeError: Timeout context manager should be used inside a task

Question

Background: I am hosting a flask server alongside a discord client

The flask server just needs to pass on messages from the client to discord and from messages from discord to the client.

I am getting the error when I call loop.run_until_complete(sendMsg(request)) I have tried wait_for in sendMsg and wait_for loop.run_until_complete()

I have looked everywhere and haven't found anything so any help would be appreciated.

Code:

import discord
import json
import os
import asyncio
from flask import Flask, request, render_template
from async_timeout import timeout
from threading import Thread
from time import sleep

client = discord.Client()
messages = []
app = Flask(__name__)

def startClient():
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    client.run('token')


#
# Discord Events
#
@client.event
async def on_ready():
    print('Discord Client Ready')

@client.event
async def on_message(message):
    global messages
    message.append(message)


#
# Flask Stuff
#
async def sendMsg(request):
    await client.send_message(discord.Object('channel id'), request.form['message'])


@app.route("/chat/", methods=['GET', 'POST'])
def chatPage():
    global messages

    if request.method == 'GET':
        return render_template('main.html')

    elif request.method == 'POST':
        loop = asyncio.new_event_loop()
        loop.run_until_complete(sendMsg(request))
        return ''

@app.route("/chat/get", methods=['GET'])
def chatGet():
    return json.dumps(messages[int(request.args['lastMessageId']):])


# Start everything
os.environ["WERKZEUG_RUN_MAIN"] = 'true'
print('Starting discord.py client')
Thread(target=startClient).start()
print('Starting flask')
app.run(host='0.0.0.0', debug=True)

Traceback:

Traceback (most recent call last):
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/mnt/c/Users/SuperKooks/Documents/Coding/HTML/kindle-discord/app.py", line 51, in chatPage
    loop.run_until_complete(sendMsg(request))
  File "/usr/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
    return future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/mnt/c/Users/SuperKooks/Documents/Coding/HTML/kindle-discord/app.py", line 39, in sendMsg
    await client.send_message(discord.Object('382416348007104513'), request.form['message'])
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/discord/client.py", line 1152, in send_message
    data = yield from self.http.send_message(channel_id, content, guild_id=guild_id, tts=tts, embed=embed)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/discord/http.py", line 137, in request
    r = yield from self.session.request(method, url, **kwargs)
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/aiohttp/client.py", line 555, in __iter__
    resp = yield from self._coro
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/aiohttp/client.py", line 197, in _request
    with Timeout(timeout, loop=self._loop):
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/async_timeout/__init__.py", line 39, in __enter__
    return self._do_enter()
  File "/home/SuperKooks/.local/lib/python3.5/site-packages/async_timeout/__init__.py", line 76, in _do_enter
    raise RuntimeError('Timeout context manager should be used '
RuntimeError: Timeout context manager should be used inside a task

Brandybrandyn

Answer 1

aiohttp.ClientSession() wants to be called inside a coroutine. Try moving your Client() initializer into any async def function

Studied

This was exactly my problem, although I'm not sure it answers the question above. – Thurnau

Than wouldn't Client() object is created every time the async def function is called? – Thirlage

Yes, it will be created every time you call it. In cases when this represents a single "session" with a server, that's fine. But if you want to keep using it, then consider putting it somewhere. Or better, reuse cookies! – Studied

Answer 2

The problem looks like it could be caused by the following:

elif request.method == 'POST':
    loop = asyncio.new_event_loop()
    loop.run_until_complete(sendMsg(request))

That creates a new event loop and runs sendMsg(request) in the new loop. However, sendMsg calls a method on the client object running in its own event loop. sendMsg should be submitted to the existing event loop that runs the client in the other thread. To accomplish that, you need to:

• expose the loop created in startClient, e.g. to a client_loop global variable;
• replace loop = asyncio.new_event_loop(); loop.run_until_complete(sendMsg(request)) with a call to asyncio.run_coroutine_threadsafe to submit the coroutine to an event loop already running in a different thread.

The submit code would look like this:

elif request.method == 'POST':
    # submit the coroutine to the event loop thread
    send_fut = asyncio.run_coroutine_threadsafe(sendMsg(request), client_loop)
    # wait for the coroutine to finish
    send_fut.result()

Beady

I have am now using aiohttp but I still got the error. I have tried this code but whenever I trigger the new code it hangs the process. – Brandybrandyn

@Brandybrandyn Have you tried debugging it? Does the sendMsg coroutine start executing? Does it finish? If it hangs, where exactly does it hang? A minimal example that reproduces the problem would also help. – Beady

Answer 3

I have a production-hardened technique to prevent both these errors:

• RuntimeError: Timeout context manager should be used inside a task,
• RuntimeError: This event loop is already running

The idea is to have a dedicated background thread that plays the event-loop, like in a traditional UI setting, however here it's not for UI messages but for API messages (i.e. requests). We start that thread once (upon import of the module, but wherever really). Furthermore, we only call asyncio_run instead of asyncio.run and asyncio_gather instead of asyncio.gather.

Here you go:

import asyncio
import threading
from typing import Awaitable, TypeVar
T = TypeVar("T")

def _start_background_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

_LOOP = asyncio.new_event_loop()
_LOOP_THREAD = threading.Thread(
    target=_start_background_loop, args=(_LOOP,), daemon=True
)
_LOOP_THREAD.start()

def asyncio_run(coro: Awaitable[T], timeout=30) -> T:
    """
    Runs the coroutine in an event loop running on a background thread,
    and blocks the current thread until it returns a result.
    This plays well with gevent, since it can yield on the Future result call.

    :param coro: A coroutine, typically an async method
    :param timeout: How many seconds we should wait for a result before raising an error
    """
    return asyncio.run_coroutine_threadsafe(coro, _LOOP).result(timeout=timeout)


def asyncio_gather(*futures, return_exceptions=False) -> list:
    """
    A version of asyncio.gather that runs on the internal event loop
    """
    async def gather():
        return await asyncio.gather(*futures, return_exceptions=return_exceptions)

    return asyncio.run_coroutine_threadsafe(gather(), loop=_LOOP).result()

Arcturus

Do you possibly have a more up-to-date version of this? Apparently the loop argument for asyncio.gather is deprecated in python 3.10. I have very basic knowledge of how asyncio works, but your code here was working great for me. – Soapbox

@EthanPosner I'll update if we upgrade to 3.10, until then, sorry. But even though it's marked as deprecated, it'll most likely still work. – Arcturus

Would you please post the full example? – Gynecoid

Answer 4

I fixed this by replacing all calls to asyncio.run with the asyncio_run below. It solved both of these errors for me:

• RuntimeError: Timeout context manager should be used inside a task
• RuntimeError: This event loop is already running

pip install nest-asyncio

import asyncio
import nest_asyncio

def asyncio_run(future, as_task=True):
    """
    A better implementation of `asyncio.run`.

    :param future: A future or task or call of an async method.
    :param as_task: Forces the future to be scheduled as task (needed for e.g. aiohttp).
    """

    try:
        loop = asyncio.get_running_loop()
    except RuntimeError:  # no event loop running:
        loop = asyncio.new_event_loop()
        return loop.run_until_complete(_to_task(future, as_task, loop))
    else:
        nest_asyncio.apply(loop)
        return asyncio.run(_to_task(future, as_task, loop))


def _to_task(future, as_task, loop):
    if not as_task or isinstance(future, asyncio.Task):
        return future
    return loop.create_task(future)

A secondary goal was to be able to think of asyncio.run as promise.resolve from the JS world, or Task.Wait from the .NET world.

Arcturus

If nest_asyncio is an external library, you should include the pip install command and the import. Also, please specify what Task is and where it should be imported from. – Levesque

@AbrahamMurcianoBenzadon. Task is from the .NET world. It's meant as an analogy, not something to directly apply/import – Arcturus

I meant in the python code isinstance(future, Task) – Levesque

worked fine except the asyncio.get_running_loop() which isn't available for python 3.6 with asyncio 3.4.3. But luckily replacing it with asyncio.get_event_loop() worked too – Fab

Answer 5

There's a really simple solution for this, turns out discord.py client has its own thread running, so you just need to add that to the coroutine that the second answer mentioned. So no need for any more loops.

1. Remove the event loop while starting client

def startClient():
    # no loops here
    client.run('token')

2. Add the inbuilt variable for discord's loop, so in /chat/ route:

elif request.method == 'POST':
    # using discord client's global
    msg = asyncio.run_coroutine_threadsafe(sendMsg(request), client.loop)
    msg.result()

That's it, hope this helps.

Despotism

Answer 6

The way I got this working is to have the client create a task and run your code in there. So

elif request.method == 'POST':
        loop = asyncio.new_event_loop()
        loop.run_until_complete(sendMsg(request))
        return ''

would become

elif request.method == 'POST':
        client.loop.create_task(sendMsg(request))

Slush