LogBack - LogStash - Add properties in the logback and send them to Logstash

Question

I'm using Logback and Logstash in a SpringBoot application.

In the logback.xml I have a property with the name of the service, and is like:

<configuration>

<include resource="org/springframework/boot/logging/logback/defaults.xml" />

<include resource="org/springframework/boot/logging/logback/console-appender.xml" />

<property name="spring.application.name" calue="service"/>

<appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>localhost:9600</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

<root level="INFO">
    <appender-ref ref="CONSOLE" />
    <appender-ref ref="stash" />
</root>

</configuration>

The Logstash conf file is like:

input{ tcp{
        port=> 9600
        host=>logstash
    }
}

filter {
grok {
match => {
  "message" =>
  "^%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:level}\s+%{NUMBER:pid}\s+---\s+\[\s*%{USERNAME:thread}\s*\]\s+%{JAVAFILE:class}\s*:\s*%{DATA:themessage}(?:\n+(?<stacktrace>(?:.|\r|\n)+))?$"
}
}
date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]
}
 mutate {
remove_field => ["@version"]
add_field => {
  "appid" => "%{[path]}"
}
add_field => {
  "levell" => "level"
}
add_field => {
  "mensage" => "message"
}
 }
   }
output{

 elasticsearch {
hosts => ["elasticsearch"]
index => "indice"
 }
   stdout{}
 } 

How can I do to add the property of application name from the logback file as a field?

Bartolomeo

Answer 1

From the logstash-logback-encoder docs:

By default, each property of Logback's Context (ch.qos.logback.core.Context), such as HOSTNAME, will appear as a field in the LoggingEvent. This can be disabled by specifying false in the encoder/layout/appender configuration.

By default your logback properties are local scope and aren't included. Try setting them to scope="context".

<property name="spring.application.name" value="service" scope="context"/>

Heartburning

Thanks @Heartburning ! The property appears in the log, but whitin "message" field. I want to have the property as field, to filter the logs with Kibana. Do you know how to do that? I tried to put this in the logastash conf file, but it didn't work: add_field => { "service" => ["@spring.application.name"] } – Bartolomeo

Answer 2

You may configure the custom fields for LogstashEncoder as follows

<appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>192.168.99.100:4560</destination>

    <!-- encoder is required -->
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
      <customFields>{"appname":"${appName}"}</customFields>
    </encoder>
</appender>

For example, for spring boot application you can use get the spring scope properties as follows

<springProperty name="appName" source="spring.application.name"/>

Or otherwise import properties from .properties file

<property  resource="application.properties" />

Corinthian

Does application.yml file work with this config? I am trying <property resource="application.yml" /> but it is not working. – Ghassan

@nikhil-kakade To use a YAML config : <property resource="application.yml" />, please take a look at : response of @granadacoder – Calendula