Grok pattern for data separated by pipe

Question

I have a logfile in which the data is separated by a pipe symbol. "|". An example is below. Does anyone know how to write a GROK pattern to extract it for logstash?

2014-01-07 11:58:48.7694|LOGLEVEL|LOGSOURCE|LOGMESSAGE

Harrietteharrigan

Answer 1

You can use gsub API to change the pipe "|" to space and the use GROK to extract it.

For example:

filter {
    grok {
            match => ["message","%{DATESTAMP:time}\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}"]
    }
}

The above configuration is worked on me with your log. Hope this can help you.

Outgrow

grok { match => ["message","%{DATESTAMP:time}\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}"] } This also will work, it worked for me – Reynoso

@ArulT, your comment has more upvotes than the answer, can you please change it to a proper answer? – Rob

Answer 2

use this filter:

it works for me. use this site to verify grok patern, https://grokdebug.herokuapp.com/

(?<date>(([0-9]+)-*)+ ([0-9]+:*)+.*)\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}

Ensconce

Answer 3

This worked for me

grok { match => ["message","%{DATESTAMP:time}\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}"] }

Reynoso

Answer 4

The LOGMESSAGE part can contain a long content. For this reason, I recommend the following usage.

%{GREEDYDATA:LOGMESSAGE}

Animalize

Use grokdebug.herokuapp.com – Animalize

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Mcvay