Spring Boot Logging and Google Cloud Platform Log Viewer
Asked Answered
F

1

8

I'm running a Spring Boot application within the Google Cloud Platform and viewing the log files viewing the Google Platform Logs Viewer. Before using Spring Boot and just using simple servlets, the logging entries would be displayed as:

Logs are grouped and show the blue information icon

Each request would be grouped and all the logging information for that request could be seen by expanding the row. However, when using Spring Boot the requests are no longer grouped and the log entries are just shown line by line. When there are multiple requests the log entries get very confusing as a result because it isn't possible to view them in a grouped way. I have my logging.properties setup in the same way:

.level = INFO
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format = [%1$tc] %4$s: %2$s - %5$s %6$s%n

The Logger is initialised in each class as:

private static final java.util.logging.Logger LOG = java.util.logging.Logger.getLogger(MyClass.class.getName());

And then the logging API is used as: LOG.info("My Message");

I don't understand why the statements are being logged differently and no longer grouped but it must have something with the way Spring Boot handles logging?

Fradin answered 7/2, 2020 at 14:51 Comment(5)
I'm also interested into this, I had some problems with stackdriver logging + app engine + spring boot in the past: #56907855 maybe it might be useful to you, but in the end I didn't have the "clean" old solutionLogistics
Thanks, glad i'm not the only one. I've tried various configurations, tried using slf4j but didn't make any difference. Ended up excluding in the pom.xml. From looking at the Google Logs Viewer you can choose filters, it is when the stdout filter is added it will cause every line to be logged and that is the only way to see the entries but they just aren't grouped. Without Spring Boot it works fine. I just don't understand what is causing it. Hopefully someone out there knows. Thanks for taking the time to reply.Fradin
I've also performed the same checks. My idea is that there's an underlying component that is using stackdriver apis. In my mind I would like to understand how app engine handles logs, I think they parse them from an output but the format is not clear. Maybe putting the exact same format and disabling stackdriver api might fix it .Logistics
Please Adam could you give some additional info please : java version of GAE runtime? java8 or java11. GAE std or flexible ? Thanks !Ouch
Hi Thierry, I am using Java 11 and standard environment. Here is my app.yaml configuration: runtime: java11 instance_class: F4 inbound_services: - warmup # Explicitly set the memory limit and maximum heap size for the Spring Boot app env_variables: JAVA_TOOL_OPTIONS: "-XX:MaxRAM=512m -XX:ActiveProcessorCount=2 -Xmx128m"Fradin
O
8

Since recent runtimes, AppEngine is evolving with a behaviour that is more and more converging with a container based approach, more "opened" as new other products (like Cloud Run for example).

This is changing a little the way we're developing with GAE, specific legacy libraries aren't available (SearchAPI...), and it is changing also how logs are managed.

We can reproduce this "nested log feature" with new java11 runtime, but we need to manage it ourself.

As official docs mentioned:

In the Logs Viewer, log entries correlated by the same trace can be viewed in a "parent-child" format.

It means, if we retrieve the trace identifier received inside X-Cloud-Trace-Context HTTP header of our request, we can then use it to add a new LogEntry by passing it as the trace identifier attribute.

This can be done by using Stackdriver Logging Client libraries

With Spring GCP

Fortunately, Spring Cloud GCP is there to make our lives easier.

You can find a sample project which implements it. Be careful, it's a AppEngine Flexible example, but it will work fine with Standard runtime. It uses Logback.

From a working Spring Boot project on GAE Java11, steps to follow are :

  • Add spring-cloud-gcp-starter-logging dependency :
<!-- Starter for Stackriver Logging -->   
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-gcp-starter-logging</artifactId>
   <version>1.2.1.RELEASE</version>
</dependency>

  • Add a logback-spring.xml inside src/main/resources folder :
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/cloud/gcp/autoconfigure/logging/logback-appender.xml" />
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml" />

    <root level="INFO">
        <!-- If running in GCP, remove the CONSOLE appender otherwise logs will be duplicated. -->
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="STACKDRIVER" />
    </root>
</configuration>
  • Enable Spring GCP logging feature, inside src/main/resources/application.properties :
spring.cloud.gcp.logging.enabled=true
  • And use LOGGER inside your code:
@SpringBootApplication
@RestController
public class DemoApplication {
    private static final Log LOGGER = LogFactory.getLog(DemoApplication.class);

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @GetMapping()
    public SomeData get() {
        LOGGER.info("My info message");
        LOGGER.warn("My warning message");
        LOGGER.error("My error message");

        return new SomeData("Hello from Spring boot !");
    }
}

Result will be in Stackdriver Logging viewer, for appengine.googleapis.com/request_log :

Stackdriver Logging Viewer

Ouch answered 12/2, 2020 at 14:56 Comment(5)
Thank you Thierry, I'll be giving this a go first thing tomorrow and see how it looks. Thank you for the detailed response, it makes sense.Fradin
Excellent, thank you Thierry! Worked perfectly, great explanation and instructions.Fradin
The Spring <-> GCP library does nice job correlating custom logs with request logs but unfortunately the request logs are not searchable via custom log entries. Meaning that if I log "Hello world" as a custom log, I can't search in stackdriver to give me requests which logged "Hello world". How would one overcome this issue?Salomo
Unfortunately the upvoted solution did not work for me (I had a similar logback config). My question there #71571234 is a bit close to this one , the problem is that the traceId is different between the request log and the app logs. Do you have any idea @thierry ? Thanks in advance !!!Polliwog
Nice solution, but may be you should add an extra steep: Inside the console.cloud.google.com select a "parent log name" -> request_log to order the logs by request. This option is next to the download button.Laural

© 2022 - 2024 — McMap. All rights reserved.