I have a simple Cloudwatch dashboard to monitor the running task count in a simple ECS cluster. AWS provides this documentation which is exactly what I want. The problem is it doesn't accurately reflect on my dashboard.
Often my desired service count is total of 3: 1 django task and 2 celery task. Most of the time this is reflected correctly in the small dashboard, but not always, and it's driving me a little crazy :\
Current configuration is by-the-books. I'm using ServiceName,ClusterName dimensions, CPUUtilization as the metric, Sample Count for the statistic and 1 minute for the period. This is how AWS says I should do it.
Yet I see this on my dashboard: https://i.sstatic.net/XiT2o.jpg
Looking directly at ECS, however, I have only 1 django and 2 celery tasks running. The Cloudfront dashboard appears to be miscalculating by doubling, even tripling sometimes the sample count value.
Even from Cloudfront when I edit this particular widget, the correct task count is displayed: https://i.sstatic.net/QKoRc.jpg
It seems like Cloudfront is getting more than one sample count in the 1 minute period, so I've tried reducing the period to 30 seconds, but that didn't seem to help.
Is 1 minute, as suggested by AWS documentation, the proper period? If not, what is the proper period?