How to set the logging level for the elasticsearch library differently to my own logging?

Question

How can I set the logging level for the elasticsearch library differently to my own logging? To illustrate the issue, I describe the module scenario. I have a module lookup.py which uses elasticsearch like this:

import logging
logger = logging.getLogger(__name__)
import elasticsearch

def get_docs():
    logger.debug("search elastic")
    es = elasticsearch.Elasticsearch('http://my-es-server:9200/')
    res = es.search(index='myindex', body='myquery')
    logger.debug("elastic returns %s hits" % res['hits']['total'])
    .
    .
.

Then in my main file I do

import logging
import lookup.py

logging.root.setLevel(loglevel(args))
get_docs()
.
.
.

I get lots of debug messages from inside the Elasticsearch object. How can I suppress them with some code in lookup.py without suppressing the debug messages in lookup.py itself? The Elasticsearch class seems to have a logger object; I I tried to set it to None, but this didn't change anything.

Marzipan

Answer 1

The following two lines have done the trick for me to suppress excessive logging from the es library.

es_logger = logging.getLogger('elasticsearch') es_logger.setLevel(logging.WARNING)

Bigwig

Answer 2

In recent (v8.5) versions of elasticsearch, the chatty POST log seems to have moved to elastic_transport:

logging.getLogger('elastic_transport.transport').setLevel(logging.CRITICAL)

Imperial

Answer 3

I have been using this:

from elasticsearch import logger as es_logger

LOGLEVEL = 50
es_logger.setLevel(LOGLEVEL)

Reimers

Answer 4

As a supplementary answer, if you have several loggers in your project, you can do this way:

import elasticsearch

es_logger = elasticsearch.logger
es_logger.setLevel(elasticsearch.logging.WARNING)

The pros:

• You have full control and make sure that the elastic search logger won't collide with other loggers
• You may set WARNING/ERROR or DEBUG levels without remembering what number will mean

The cons:

• You are dependent on internal logger implementation of elastic search

Vescuso

Answer 5

None of the answers worked for me. I dived into elasticsearch (8.11.1) and found the spot that logs, and pulled out the logger name. Do this:

es_logger = logging.getLogger("elastic_transport.transport")
es_logger.setLevel(logging.WARNING)

Iraqi

Thank you a billion times. This is the only solution that worked for me. I wish it was the first result when searching for this issue, so that I wouldn't waste an hour trying to find a solution. – Demers

Answer 6

I had the same issue and previous solutions didn't work. I assume the name of the Elasticsearch logger changed again, so I used the information in this Python documentation to find a solution.

By avoiding "basicConfig" and using FileHandler, there was no more conflict.

Sklar