CloudWatch Logs Filter case insensitive multiple terms or connected

Question

I'm just trying to create an alarm based on CloudWatch Logs Filter which triggers on multiple terms (or connected, not and) and is case insensitive

Using "error warning" as pattern is not working

I'm looking for filter pattern reacting to all of the following errors and warnings:

ERROR: first sample
Error: second sample
error: third sample
{ ERROR: "fourth sample"}
{type: "error"}
WARNING: SOMETHING BAD!
{ WARNING: "fifth sample"}

Eskimo

Answer 1

Per the AWS Documentation concerning Filter and Pattern Syntax, you cannot use "error warning" to capture an "OR" relationship because:

• You can specify multiple terms in a metric filter pattern, but all terms must appear in a log event for there to be a match.

Or in other words, CloudWatch Log metric filters expect an "AND" relationship.

Likewise:

• Metric filters are case sensitive.

So you'll be unable to achieve this with a single filter. You'll need a filter for each case-sensitive permutation of "error" and "warning" that you expect to write to Cloudwatch Logs.

In order to set a single alarm on all of these filters, simply configure each filter to use the same CloudWatch metric. Here's an example from the AWS Console where each of my metric filters are targeted towards my LogMetric/test metric:

I can then simply create a CloudWatch alarm based on the LogMetric/test metric to alarm on the sum of these distinct metric filters.

Rafaelita

Answer 2

If you need to filter upon some strings you can OR them as follows:

?"String1" ?"String2" 

and so on. Try it.

Gal

This is the best solution! In my case I need to find the string "erro" (portuguese) or "error" (english). So the construction ?"ERRO" ?"erro" ?"Erro" solves my problem, as it allows us to search for several cases - workaround for "case insensitive" - and for part of the desired string. The connector between the terms will be "OR". Thanks! 🤗 – Weidar

I get this error in form: Invalid character(s) in term '?' – Algebraist

Even though its been years, it still works for me. Can you post you entire search term @AlexFreshmann – Gal

Answer 3

Let's present two ways to solve the problem...

I - Using filters (Log groups)

1. Go to AWS CloudWatch;
2. Click on "Log groups" ("Logs");
3. Search for the desired log group;
4. Select the desired log group;
5. Click on "Search log group";
6. Apply the desired filter in the relevant field and other desired search parameters.

FILTER EXAMPLE

?"ERROR" ?"Error" ?"error" ?"EXCEPT" ?"Except" ?"except"

NOTE: Allows you to search multiple cases - workaround for "case insensitive" - and by desired string part. The connector between the terms will be "OR".

II - Using queries (Logs Insights)

1. Go to AWS CloudWatch;
2. Click on "Logs Insights" ("Logs");
3. Search for the desired log group;
4. Select the desired log group;
5. Insert your query;
6. Apply other desired search parameters;
7. Click on "Run query".

QUERY EXAMPLE

fields @timestamp, @message
| filter @message like /(?i)(error|except)/
| sort @timestamp desc
| limit 20

NOTE: Allows you to search case insensitive and by desired string part. The connector between the terms will be "OR".

Thanks! 🤗

[Ref(s).: https://mcmap.net/q/758451/-cloudwatch-logs-filter-case-insensitive-multiple-terms-or-connected , https://bneijt.nl/blog/cloudwatch-case-insensitive-like-filter/ , https://mcmap.net/q/402897/-how-do-we-sort-cloudwatch-stream-logs-by-39-most-recent-39-in-aws-console ]

Weidar

Answer 4

In some simple cases, it might help to use regex: %[Ee]rror%

Guttersnipe