Can S3 Select search multiple objects?

Question

I'm testing out S3 Select and as far as I understand from the examples, you can treat a single object (CSV or JSON) as a data store.

I wanted to have a single JSON document per S3 object and search the entire bucket as a 'database'. I'm saving each 'file' as <ID>.json and each file has JSON documents with the same schema.

Is it possible to search multiple objects in a single call? i.e. Find all JSON documents where customerId = 123 ?

Bethannbethanne

We have added a script which can loop through the folder and run the S3 Select on files and write it to a file: link.medium.com/6vKrErw9qR – Ticktack

Answer 1

It appears that Amazon S3 Select operates on only one object.

You can use Amazon Athena to run queries across paths, which will include all files within that path. It also supports partitioning.

Samons

Answer 2

Simple, just iterate over the folder key in which you have all the files and grab the key and use the same to leverage S3 Select.

Gilus

This is not what is asked. – Kerekes