How do I disable log messages from the Requests library?

Question

By default, the Requests python library writes log messages to the console, along the lines of:

Starting new HTTP connection (1): example.com
http://example.com:80 "GET / HTTP/1.1" 200 606

I'm usually not interested in these messages, and would like to disable them. What would be the best way to silence those messages or decrease Requests' verbosity?

Enochenol

Related: nginx and gunicorn – Cameliacamella

Answer 1

I found out how to configure requests's logging level, it's done via the standard logging module. I decided to configure it to not log messages unless they are at least warnings:

import logging

logging.getLogger("requests").setLevel(logging.WARNING)

If you wish to apply this setting for the urllib3 library (typically used by requests) too, add the following:

logging.getLogger("urllib3").setLevel(logging.WARNING)

Enochenol

I have same issue with pysimplesoap, and this answer help me save my day – Bath

You could combine the two lines like this: logging.getLogger('requests').setLevel(logging.WARNING) – Leith

@JasperPoppe Thanks for the suggestion, implemented. – Enochenol

I had to add this line for the "urllib3" logger to suppress requests log messages. – Herold

I was need to import logging; logging.getLogger("urllib3").setLevel(logging.WARNING), too. Logger for "requests" doesn't prevent these messages. – Varrian

For some reason when using the requests library in python3 you have to do getLogger("urllib3") to suppress the messages. – Faerie

@Robru Sounds as if requests is basing itself on urllib3 these days quite simply, which does logging of its own. That's how it goes :) – Enochenol

Is there a way to make change the level of the logging from requests to DEBUG instead of INFO? I would like to keep the messages, but only at the DEBUG level (so they are only in my offline debug log, and not on the screen). – Cosmic

@FrankMeulenaar I would pose that as an independent question. Anyhow, I guess what I would do is configure the requests stream handler outputting to screen to ignore anything below or equal to INFO. – Enochenol

I set it to CRITICAL but I'm still getting the same log message: 2017-08-05 15:17:13,334 ERROR -- : Certificate did not match expected hostname: www.improving-autonomy.org. Certificate: {'subjectAltName': [('DNS', '*.wordpress.com'), ('DNS', 'wordpress.com')], 'subject': ((('commonName', u'*.wordpress.com'),),)} Any help will be highly appreciated – Forgery

Solved. Full path was needed: logging.getLogger("requests.packages.urllib3").setLevel(logging.CRITICAL) – Forgery

Current versions of the requests library do not vendor urllib3 any more and so doesn't log anything. You only need to ask the urllib3 library to not propagate: logging.getLogger("urllib3").propagate = False, or increase the logging level if you still need to see warnings or critical messages (only warnings are issued currently). – Stringfellow

Answer 2

In case you came here looking for a way to modify logging of any (possibly deeply nested) module, use logging.Logger.manager.loggerDict to get a dictionary of all of the logger objects. The returned names can then be used as the argument to logging.getLogger:

import requests
import logging
for key in logging.Logger.manager.loggerDict:
    print(key)
# requests.packages.urllib3.connectionpool
# requests.packages.urllib3.util
# requests.packages
# requests.packages.urllib3
# requests.packages.urllib3.util.retry
# PYREADLINE
# requests
# requests.packages.urllib3.poolmanager

logging.getLogger('requests').setLevel(logging.CRITICAL)
# Could also use the dictionary directly:
# logging.Logger.manager.loggerDict['requests'].setLevel(logging.CRITICAL)

Per user136036 in a comment, be aware that this method only shows you the loggers that exist at the time you run the above snippet. If, for example, a module creates a new logger when you instantiate a class, then you must put this snippet after creating the class in order to print its name.

Stencil

Thank you, this helped me silence urllib3 log messages when using boto3. The logger in such case is botocore.vendored.requests.packages.urllib3, so I used this: logging.getLogger("botocore.vendored.requests.packages.urllib3").setLevel(logging.WARNING) and I finally got rid of the messages. – Spreadeagle

Many thanks for this! Altering the print criteria allowed me to isolate that python-elasticsearch was the culprit in my case. – Linetta

Be aware that this will not work when modules create their loggers inside their class you'd call later, like the APScheduler does when you call BackgroundScheduler.BackgroundScheduler(). – Disequilibrium

@user136036: logger objects are singletons, it doesn't matter if you or the library get to create them first. If you use the exact same name as the library uses, it will work. – Stringfellow

I think they are saying that if you list the loggers before a library has created its logger, then it won’t be listed. Which is correct. – Stencil

This helped me find the module that was writing log messages with request information that I wanted to get rid off (azure). Thank you so much! – Cowpuncher

This is a lifesaver. I was trying to disable logging by setting propagate = False and could not get it working no matter what. Finally, this helped me to get logger name as urllib3.connectionpool as urllib3 and connectionpool would not work. – Nieman

Answer 3

import logging
urllib3_logger = logging.getLogger('urllib3')
urllib3_logger.setLevel(logging.CRITICAL)

In this way all the messages of level=INFO from urllib3 won't be present in the logfile.

So you can continue to use the level=INFO for your log messages...just modify this for the library you are using.

Lozano

I suggest using setLevel(logging.WARNING) to log also possible warning and error messages. – Minster

Answer 4

import logging

# Only show warnings
logging.getLogger("urllib3").setLevel(logging.WARNING)

# Disable all child loggers of urllib3, e.g. urllib3.connectionpool
logging.getLogger("urllib3").propagate = False

Cameliacamella

Answer 5

For anybody using logging.config.dictConfig you can alter the requests library log level in the dictionary like this:

'loggers': {
    '': {
        'handlers': ['file'],
        'level': level,
        'propagate': False
    },
    'requests.packages.urllib3': {
        'handlers': ['file'],
        'level': logging.WARNING
    }
}

Shona

@SebastianWagner Django uses dictConfig under the hood. – Capsulize

Thank you so much! This is pretty good. One place to rule all of the library logs!! :) – Pascia

Answer 6

Let me copy/paste the documentation section which it I wrote about week or two ago, after having a problem similar to yours:

import requests
import logging

# these two lines enable debugging at httplib level (requests->urllib3->httplib)
# you will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# the only thing missing will be the response.body which is not logged.
import httplib
httplib.HTTPConnection.debuglevel = 1

logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

requests.get('http://httpbin.org/headers')

Lentz

What's the point of being more specific than just 'requests' though, from a practical POV? – Enochenol

But what do you gain by calling logging.getLogger("requests.packages.urllib3") instead of logging.getLogger("requests"), considering that you want to affect the logging of the requests library? – Enochenol

Do you mean that you wish to enable logging within requests.packages.urllib3? If so, you're answering the wrong question. – Enochenol

@Enochenol It's up to you if you want to disable or enable them, I only put the code that fully controls this :) – Lentz

I think you've misunderstood the question's scope. – Enochenol

Answer 7

Setting the logger name as requests or requests.urllib3 did not work for me. I had to specify the exact logger name to change the logging level.

First See which loggers you have defined, to see which ones you want to remove

print(logging.Logger.manager.loggerDict)

And you will see something like this:

{...'urllib3.poolmanager': <logging.Logger object at 0x1070a6e10>, 'django.request': <logging.Logger object at 0x106d61290>, 'django.template': <logging.Logger object at 0x10630dcd0>, 'django.server': <logging.Logger object at 0x106dd6a50>, 'urllib3.connection': <logging.Logger object at 0x10710a350>,'urllib3.connectionpool': <logging.Logger object at 0x106e09690> ...}

Then configure the level for the exact logger:

   'loggers': {
    '': {
        'handlers': ['default'],
        'level': 'DEBUG',
        'propagate': True
    },
    'urllib3.connectionpool': {
        'handlers': ['default'],
        'level': 'WARNING',
        'propagate' : False
    },

Christlike

where are these levels set? – Rog

I have them in Django settings, in base.py. Where to put them of course depends on your project setup. – Christlike

Answer 8

If You have configuration file, You can configure it.

Add urllib3 in loggers section:

[loggers]
keys = root, urllib3

Add logger_urllib3 section:

[logger_urllib3]
level = WARNING
handlers =
qualname = requests.packages.urllib3.connectionpool

Halsy

This is a perfectly valid answer for people who use configuration file. Not sure why it got so many down vote? – Reformism

Answer 9

This answer is here: Python: how to suppress logging statements from third party libraries?

You can leave the default logging level for basicConfig, and then you set the DEBUG level when you get the logger for your module.

logging.basicConfig(format='%(asctime)s %(module)s %(filename)s:%(lineno)s - %(message)s')
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

logger.debug("my debug message")

Cammycamomile

Answer 10

I'm not sure if the previous approaches have stopped working, but in any case, here's another way of removing the warnings:

PYTHONWARNINGS="ignore:Unverified HTTPS request" ./do-insecure-request.py

Basically, adding an environment variable in the context of the script execution.

From the documentation: https://urllib3.readthedocs.org/en/latest/security.html#disabling-warnings

Combined

Answer 11

Kbrose's guidance on finding which logger was generating log messages was immensely useful. For my Django project, I had to sort through 120 different loggers until I found that it was the elasticsearch Python library that was causing issues for me. As per the guidance in most of the questions, I disabled it by adding this to my loggers:

      ...
      'elasticsearch': {
          'handlers': ['console'],
          'level': logging.WARNING,
      },     
      ...

Posting here in case someone else is seeing the unhelpful log messages come through whenever they run an Elasticsearch query.

Linetta

Answer 12

In my case what helped me (python 3.7)

import http.client as http_client
http_client.HTTPConnection.debuglevel = 0

Porkpie

Answer 13

simple: just add requests.packages.urllib3.disable_warnings() after import requests

Funch

I did not find this method in my version. Disabling warnings is excessive, as the annoying messages are level INFO. – Cerallua