How to consolidate Multiline logs in syslog

I am feeding my var/log/message using rsyslog from a tool. The exception appears in multiple lines (on line per message) rather than logging it as one multi-line message.

I would like my /var/log/message to look like the catalina.out message. Is there any way I can achieve this?

The catalina.out with one multi-line message:

at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

The var/log/message which is splitted into multiple messages:

2014-02-20T06:21:32.006782+00:00 something148-084-115 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
2014-02-20T06:21:32.006782+00:00 something148-084-115 at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
2014-02-20T06:21:32.006784+00:00 something148-084-115 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
2014-02-20T06:21:32.006784+00:00 something148-084-115 at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)
2014-02-20T06:21:32.006786+00:00 something148-084-115 at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)

The log4j.xml config:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
   <appender name="console" class="org.apache.log4j.ConsoleAppender">
      <param name="Target" value="System.out" />
      <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%d{ISO8601} abc: [component=&quot;XYZ&quot; priority=&quot;%p&quot; thread=&quot;%t&quot;] %c.%M:%L - %m%n" />
      </layout>
   </appender>
   <appender name="syslog" class="org.apache.log4j.net.SyslogAppender">
      <param name="syslogHost" value="localhost" />
      <param name="threshold" value="INFO" />
      <param name="facility" value="LOCAL0" />
      <param name="facilityPrinting" value="false" />
      <layout class="org.apache.log4j.PatternLayout">
         <param name="ConversionPattern" value="abc: [component=&quot;XYZ&quot; priority=&quot;%p&quot; thread=&quot;%t&quot;] %c.%M:%L - %m%n" />
      </layout>
   </appender>
   <root>
      <priority value="info" />
      <appender-ref ref="console" />
      <appender-ref ref="syslog" />
    </root>
    <logger name="org.springframework">
      <level value="warn" />
    </logger>
    <logger name="org.hibernate">
       <level value="warn" />
    </logger>
</log4j:configuration>

As this question was asked 8 years ago, i'm not going to bother to answer this specific question, but try to make it easier for anyone coming across this post just now.

As of rsyslog version 8.10, they added the ability to use the imfile module to process multi-line messages from a text file. The imfile module enables rsyslog to convert any text file into a stream of syslog messages. You can include a startmsg.regex parameter that defines a regex pattern that rsyslog will recognize as the beginning of a new log entry. If rsyslog detects the pattern, it will aggregate all subsequent log entries into the same event until it finds another matching line.

By default, rsyslog can send and receive log messages up to 8 KB. Multi-line messages can potentially be much larger than this. In order to make sure rsyslog properly handles large multi-line messages, you can increase the maximum message size to 64 KB by adding the following to the (very) top of your rsyslog.conf

$MaxMessageSize 64k

To add the imfile module, append the following:

module(load="imfile" mode="inotify")

Traditionally, imfile used polling mode, which is much more resource-intense (and slower) than inotify mode. It is suggested that users turn on “polling” mode only if they experience strange problems in inotify mode.

After that, define the file, as well as the regex, which you want to import the log messages from.

input(type="imfile" File="/var/log/local0.log" 
    startmsg.regex="^[0-9]{4}-[0-9]{2}-[0-9]{2}")

After that you can redirect the input as desired, e.g.

# Local 
action(type="omfile" file="/var/log/processed/local0.log")

# Syslog over UDP
action(type="omfwd" target="192.168.0.1" Port="514" Protocol="udp")

If you want to do this with multiple files or you want to aggregate the input to one file, it's (probably) for the best, if you create a ruleset. For more information see the rsyslog documentation.

Note: This regex was working for my specific use-case. It depends on the template you're using; this should work if you're not providing a template and therefore are using the standard syslog format.

If this should not be the case, then analyze your logs and create an expression that fits your use-case.

Recommended topics

Hot tags