log JSON queries built through ElasticSearch High Level Java Client for debugging?

Question

I use ElasticSearch High-Level Client Java API in my Spring Boot application. I want to log the queries built using High-Level client API for debugging purposes.

My question is what kind of settings required in my application.properties file to turn on JSON queries built from my application?

I tried out the following properties to the application.properties file. However, it does not print the JSON queries built using various query builders.

logging.level.org.elasticsearch.client=TRACE
logging.level.org.elasticsearch.client.sniffer=TRACE
logging.level.org.elasticsearch=TRACE

Bedeck

Thank you Opster Elastisearch Ninja for providing the solution. – Bedeck

those log configuration setting are quite good because I was able to get the HTTP request – Windowsill

Answer 1

You can simply log the queries built using the rest-high level client, using the below code in your logger.

You can also have control of which types of queries you want to log and what types of levels (TRACE, INFO, DEBUG) you want to set in specific cases.

Code to get and log the search JSON

SearchRequest searchRequest = new SearchRequest("employee").source(sourceBuilder);
log.info("Search JSON query: {}", searchRequest.source().toString());

The first line, is used to create a search request and second line is used to print the search JSON, Note searchRequest.source().toString()) which is used to get the search JSON string.

Let me know if you face any issue, I do this all the time using the rest-high-level client.

Augsburg

Thank you @Opster Elasticsearch Ninja. This is what I was looking for. – Bedeck

Answer 2

elasticsearch logging doc seems too ambiguous, but it mentions tracer:

Enable trace logging for the tracer package to have such log lines printed out.

If you dive into the es client code, there is a class called org.apache.http.util.EntityUtils.RequestLogger, which logs all the requests and responses detail into a logger named tracer:

private static final Log tracer = LogFactory.getLog("tracer");

In method logResponse, you can see that it logs debug info into normal package logger, trace info into tracer logger.

So the right way to show request & response trace info is to configure a logger named tracer, and enable TRACE level for it.

Using logback.xml for example:

    <appender name="ES_REQ_RES_TRACER" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>logs/es-trace</file>
        <append>true</append>
        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <fileNamePattern>logs/es-trace.%d{yyyy-MM-dd}.%i</fileNamePattern>
            <maxHistory>3</maxHistory>
            <maxFileSize>500MB</maxFileSize>
        </rollingPolicy>
        <encoder>
            <pattern>%date %level [%thread] %logger{10} [%file:%line] %msg%n</pattern>
        </encoder>
    </appender>

    <logger name="tracer" additivity="false" level="trace">
        <appender-ref ref="ES_REQ_RES_TRACER" />
    </logger>

Now you can find trace details in logs/es-trace file. It will contains a curl request and a json response.

TRACE level for tracer logger can also be configured in application.properties if you use spring boot:

logging.level.tracer=TRACE

Demirep

Answer 3

Spring Data Elasticsearch has guides on how to do client logging.

To see what is actually sent to and received from the server Request / Response logging on the transport level needs to be turned on as outlined in the snippet below.

Enable transport layer logging

<logger name="org.springframework.data.elasticsearch.client.WIRE" level="trace"/>

The above applies to both the RestHighLevelClient and ReactiveElasticsearchClient when obtained via RestClients respectively ReactiveRestClients.

https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.clients.logging

Olympian