Loki parse JSON and filter by field

Question

I'm logging a JSON which is being shown as a detected field log in Grafana:

Screenshot of detected fields

Now I want to filter the logs by level name, but I can't figure out how...

I thought something like this would work, but doesn't, no results are returned:

{app="hello"} | json | line_format "{{.log}}" | levelname="ERROR"

What am I missing?

Oleum

Answer 1

Try piping that bit of line_format to json again, like so:

{app="hello"} | json | line_format "{{.log}}" | json | levelname="ERROR"

Molli

Answer 2

Everything is Ok with the LogQL, but something is wrong with your JSON log line. You should have several detected fields (like "asctime", "created", "filename", etc) instead of just the "log" one, and you also should have the "Log labels" section.

Maybe you should start verifying why the quotes are escaped with ""?

See a working example following:

Ctenidium

I'm logging through python's logging lib and applying a custom formatter that json.dumps the log information. So, in the Detected fields I have a log key with, for example: "{\"asctime\": \"2022-04-29 09:28:37,817\", \"created\": 1651224517.8172572, \"filename\": \"main.py\", \"funcName\": \"root\", \"levelname\": \"ERROR\", \"levelno\": 40, \"lineno\": 30, \"module\": \"main\", \"msecs\": 817.2571659088135, \"message\": \"oh no an error\", \"name\": \"root\"}\n". Should I be doing this a different way? any pointers? – Oleum

I guess the question then is how do I format the log so its fields get detected? – Oleum

Definitively Loki isn´t understanding this is a JSON and unfortunately I don't know how to fix it. Try to get rid of these "\". – Fernandafernande

Answer 3

You don't need to unpack JSON from log line in order to filter by some JSON field in Grafana Loki. It is more efficient filtering JSON log lines directly:

{app="hello"} |= `"levelname": "ERROR"`

This query applies |= filter to JSON log line.

P.S. it is even more efficient using LogsQL for filtering on rarely occured values:

_stream:{app="hello"} `"levelname": "ERROR"`

(I'm the author of LogsQL).

Ericaericaceous