AWS CloudWatch Logs custom metrics not showing up in GUI

Question

I'm adding custom filter metrics to different logs on CW Logs. Some of them were added just fine and can be added to graphs and alarms. Some other just disappear after adding no matter how often I try to add them. I understand that there is a delay involved but even after hours I'm not able to use them.

Process seems straight forward as I'm following the guides from AWS. Is there anything I'm missing?

Phagocyte

#44951713 does this help you? – Sidestep

Answer 1

For others that may come across this, some times it can take a few days for the metrics to show up if they're far into the past:

Data points with time stamps from 24 hours ago or longer can take at least 48 hours to become available for GetMetricData or GetMetricStatistics from the time they are submitted. Data points with time stamps between 3 and 24 hours ago can take as much as 2 hours to become available for for GetMetricData or GetMetricStatistics.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_PutMetricData.html

Caretaker

It turns out to be worse than that: "Metrics that have not had any new data points in the past two weeks do not appear in the console." (SO answer) – Motorist

@Motorist Oh wow, I did not know that was a thing lol. I guess it's important to keep your metrics flowing in even if you have nothing to report? – Caretaker

Yes indeed. There is probably some technical justification for this behaviour on AWS's side, but from a user perspective it's confusing. It only adds to my impression of CloudWatch being a very clunky tool... – Motorist

I would wager it's more of a cost reduction passive consequence. The CloudWatch service as a whole deals with SO many metrics every second it's hard to imagine. – Caretaker