How to disable default log messages from lambda in python

Question

I have an AWS Lambda function written in python, and i need only the messages I log in CloudWatch Logs. I have tried the example given in watchtower, but it still didn't work.

START RequestId: d0ba05dc-8506-11e8-82ab-afe2adba36e5 Version: $LATEST
(randomiser) Hello from Lambda
END RequestId: d0ba05dc-8506-11e8-82ab-afe2adba36e5
REPORT RequestId: d0ba05dc-8506-11e8-82ab-afe2adba36e5
Duration: 0.44 ms Billed Duration: 100 ms Memory Size: 128 MB   Max Memory Used: 21 MB*

From the above I only need (randomiser) Hello from Lambda to be logged in CloudWatch, without the START, END and REPORT lines.

Xenophobe

What you want can't be done. You are always going to get those default log messages as that is part of the base AWS Lambda functionality that cannot be changed. – Somewhere

Ck, is it possible to send only required logs to a specific group in cloud watch logs in this case, i would like to see only (randomiser) Hello from Lambda in that group – Xenophobe

@Xenophobe did you find a way to do this? – Fumikofumitory

@ablemike, unfortunately i am not able find a way for this. Please help me out if any ways. – Xenophobe

@Xenophobe I found out that it is not possible. To see the logs this way, you need to use a filter. AWS will always put their logs in and it can not be turned off. – Fumikofumitory

Answer 1

If you have logs enabled, you are always going to get the default logs. No way you can disable them.

However there might be cases where you want one specific Lambda function to not send logs at all. You can solve this by creating a new role specifically for that Lambda function, and not have the logging permission there.

FWIW, if you need to toggle between logging and no logging frequently, you can have a policy file as the following.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": [ "arn:aws:logs:*:*:*" ] } ] }

and change the "Deny" to "Allow" when you require logging.

Tokenism

Do you know if this saves CloudWatch API calls? AWS charges for those. – Ghent

@Ghent Oh yes, absolutely does. Saved us hundreds of $$. – Tokenism

Answer 2

As of November 2023 all answers here are outdated - you can now configure the log level of lambdas in a much more fine grained way than before, specifically you can adjust the System log level to not generate these Start/Stop messages. The only downside is that you have to log in JSON format unfortunately. More details are available here: https://aws.amazon.com/blogs/compute/introducing-advanced-logging-controls-for-aws-lambda-functions/

Connected

Answer 3

In the AWS Lambda configuration you'll have a CloudWatch trigger configured so that the lambda is triggered by new log entries in CloudWatch. In that trigger configuration, you can specify a filter pattern, and - if you do - only those log lines that match the filter will be forwarded to your lambda.

The caveat (according to https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-syntax) seems to be that the filter operates on JSON data only, I have not found a filter that operates on plain text (though, if you put your log message in quotes, it's potentially a valid JSON string and can be matched by the filter.

Cereal

Answer 4

There is no direct way to disable these logs. However, a simple workaround is to remove the CloudWatch Logs permission from the Lambda execution role. Lambda function uses this role to access other AWS services, if you remove CloudWatch permission it will not be able to push logs to CloudWatch.

Note: if you do this you will not able to push any logs from lambda to CloudWatch

Keeping

I need logs from lambda but dont require those default log messages from lambda. – Xenophobe