gsutil / gcloud storage file listing sorted date descending?

Question

Is there no way to get a file listing out from a Google Cloud Storage bucket that is sorted by date descending? This is very frustrating. I need to check the status of files that are uploaded and the bucket has thousands of objects.

gsutil ls does not have the standard linux -t option.

Google cloud console also lists it but does not offer sorting options.

Harberd

Answer 1

I use this as a workaround:

gsutil ls -l gs://[bucket-name]/ | sort -k 2

This outputs full listing including date as the second field, sort -k 2 then sorts by this field.

Kellar

This is best answer to me. – Insurer

Short explanation of why this works: The second field in the ls output is a timestamp, -k 2 sorts by this field. – Chthonian

One downside to be aware of for large buckets: it has to download the entire list of items before the sorting can be applied. – Bifoliate

This does alphabetical sorting, so if you want to sort by size, this does not work. – Trochelminth

This sorts by oldest first how do I reverse it? – Greasy

@NathanMcKaskle I tried sort -r -k 2 but the issue is that the "TOTAL: X objects..." line will show up first. Ended up doing sort -k 2 | tail -n 2 | head -n 1. – Nostrum

Is there a way to delete older files from the list. Say retaining only latest 5 at any given moment – Dowable

@Dowable you should be able to just use head -5 or tail -5 depending on the order of your sort. e.g. sort -k 2 | tail -5 – Wichita

Answer 2

The only ordering supported by GCS is lexicographic.

As a workaround, if it's possible for you to name your objects with a datestamp, that would give you a way to list objects by date.

Landowska

I'm guessing this is by design? To allow infinitely long directories / buckets? – Harberd

I've been looking for official documentation of this feature (lexicographic object list ordering) and have been unable to find it. Are there any links that you know of? – Organizer

The ordering must be deterministic since they allow paging of results, it's just not necessarily alphabetic (or controllable?). I'd love to see some official documentation on this as well as I haven't yet been able to find it. – Pokpoke

I too was looking for official documentation about this behavior. I found this reference: cloud.google.com/storage/docs/listing-objects "This page shows you how to list the objects stored in your Cloud Storage buckets, which are ordered in the list lexicographically by name". However, it's not on the official API page: cloud.google.com/storage/docs/json_api/v1/objects/list – Defaulter