Is there a way to paginate AWS CloudWatch Logs Insights

Question

is there a way to paginate AWS CloudWatch Logs Insights? Thank you.

I was thinking something like below but that doesn't work

fields @message | page 2 | limit 20

EDIT:

I am currently accessing the logs via the Logs Insight

In my app, I am accessing it via AWS.Logger.Serilog. But I can move to a different approach (e.g. raw API call) if that would allow me to paginate the logs result.

I have tried browsing through the AWS CloudWatch documentation, google, and stack overflow. The problem I'm trying to overcome is how to paginate the logs so it won't load all the logs given a query.

Probation

Please edit your question to provide more information. For example, how are you accessing the logs (via the console, via API calls)? What problem are you trying to overcome? What have you tried so far? – Auger

@JohnRotenstein please see edit. thanks – Probation

Did you ever figure this one out? I am trying to achieve the same thing with the Java SDK. – Ting

Answer 1

There's not a native way to do this with the CloudWatch Insights Query Syntax.

You could achieve this in your application logic by using the last element in the current result set's sort field value as a next item token.

For example, your query uses @timestamp as the sort field. To fetch the next page, you could append filter (@timestamp < {value}) to the new query, where {value} is the timestamp of the previous query result's last item.

Ulani

"where" is not a valid for log insights – Nebulosity

@Nebulosity thanks, updated to use the correct filter syntax – Ulani

Answer 2

Filter by timestamp query on AWS Cloudwatch Logs Insights has solution on how to filter on @timestamp on milliseconds even. So you can build pagination with that by doing a

| fields tomillis(@timestamp) as millis
| filter millis < {time_stamp_of_last_returned_log_line} 

Nebulosity