aiohttp - exception ignored message

Question

I'm running the following code which makes 5 requests via aiohttp:

import aiohttp
import asyncio

def fetch_page(url, idx):
    try:
        url = 'http://google.com'
        response = yield from aiohttp.request('GET', url)

        print(response.status)
    except Exception as e:
        print(e)

def main():
    try:
        url = 'http://google.com'
        urls = [url] * 5

        coros = []
        for idx, url in enumerate(urls):
            coros.append(asyncio.Task(fetch_page(url, idx)))

        yield from asyncio.gather(*coros)
    except Exception as e:
        print(e)

if __name__ == '__main__':
    try:
        loop = asyncio.get_event_loop()
        loop.run_until_complete(main())
    except Exception as e:
        print(e)

Output:

200
200
200
200
200
Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in:

Note: There is no additional information as to what/where the exception is.

What's causing this and are there any tips to debug it?

Genesisgenet

Note that wrapping whole functions in try: ... except Exception is not best practice. What are you running this code in - is the "Exception ignored" message from your IDE? – Revolution

I added those to try to catch the error - the "Exception ignored" is actually printed without the try... except present. I'm thinking that something in aiohttp or asyncio is muting out the exceptions. I'm running this in a script executed via terminal in Python 3.4. – Genesisgenet

Looks like this might be related to this python bug. I'm not sure why an unraisable exception is occurring in the first place, though; I can't reproduce it on my system. – Estabrook

@Genesisgenet Also see the comments at the end of this bug where this same error is mentioned when exiting an asyncio application. Are you using Python 3.4.3? – Estabrook

Hmm, that's strange indeed. I'm using 3.5.0a1. Glad it appears to be unrelated to my code. – Genesisgenet

@Genesisgenet Well, according to the comments it probably means there's a task that isn't complete at the end of your program, but that doesn't seem to be the case here. Try adding response.close() at the end of fetch_page, and loop.close() after the loop.run_until_complete(..) line, and see if that makes a difference. – Estabrook

@Estabrook response.close() fixed it. Will you add that as an answer so I can accept? – Genesisgenet

@Estabrook Even I am facing the same issue though I am not using asyncio or aiohttp. It occurs when I am trying to allocate a class instance to a dictionary as a value to an integer key. Any idea how to stop getting the error. – Incorporated

Answer 1

I'm not exactly sure why, but it seems that leaving the aiohttp.ClientResponse object open is causing an unraisable exception to be thrown when the interpreter exits. On my system, this results in warnings like this, rather than "Exception ignored in" messages:

sys:1: ResourceWarning: unclosed <socket object at 0x7f44fce557a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f44fce55718>
sys:1: ResourceWarning: unclosed <socket object at 0x7f44fcc24a78>
sys:1: ResourceWarning: unclosed <socket object at 0x7f44fcc248c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f44fcc24958>
sys:1: ResourceWarning: unclosed <socket object at 0x7f44fcc249e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f44fcc24b08>

In any case, you can fix it by explicitly closing the ClientResponse object at the end of fetch_objects, by calling response.close().

Estabrook

Answer 2

I had a similar problem but using the Context class and its method request, you have to not forget to call shutdown() method on your Context object at the end to avoid the Exception ignored in: message. As the shutdown method is async and I was using python 3.4 the call is done using yield from.

Metaphrase