Aws Batch not logging to Cloudwatch

Question

I have a docker image through which I make an aws batch job definition. Within the docker I am printing some statements. The job runs and there's a success message. But the logs aren't there in Cloudwatch. It says that no events found. Not even the start and endtime of the job.

Cloudwatch logs

I have set up the environment variable ECS_AVAILABLE_LOGGING_DRIVERS = awslogs as well but I still don't see any logs.

What additional configuration are required ?

Gid

Unlike Lambda, which logs start/end of the job to CloudWatch Logs, Batch will only log output from the container to CloudWatch Logs. Likely your job is simply not logging to STDOUT. You shouldn't have to set any additional configuration to make logs happen - Batch should handle that. – Scent

@LukeWaite OP says that they are printing statements in their docker, this suggests they are either logging to STDOUT or STDERR, both of which should go to CloudWatch. – Bellwether

What about permissions? Does your docker has permissions to write to CW logs? Is it running on an instance or ECS? – Bangtail

Answer 1

For Using CloudWatch Logs with container instances you need the following:

• CloudWatch Logs IAM Policy
• Installing and configuring the CloudWatch agent on your Container

the steps could be found here

Tiv

The ECS containers were setup by AWS Batch and the IAM Policy was added by the wizzard. – Bellwether

you still need to install and configure the cloudwatch agent in your docker image – Tiv

see this also aws.amazon.com/blogs/devops/… – Tiv

Turns out the problem was that log shipping was delayed by a few hours. Will create a new question based on that...awarding you the rep anyway, since it is probably the correct answer. – Bellwether

I also have an issue where a batch job fails, and it takes from several minutes to at least half an hour for the logs to appear in cloudwatch. Is there any way to hasten this process? – Hamal