does aws s3 select work with multiple files?

Question

I'm using S3 select to query a object from a file in S3 bucket but wondering if I can do it with multiple files or if its limited to just one. I wrote one query from a file using aws lambda but when I tried writing another one for a different file it gave me a error message.

Leeland

I am having similar issue. I am capturing incoming messages from a temperatures sensor through AWS IoT in to an s3 bucket. Each s3 object is a new temperature reading. Some thing like aws s3api select-object-content --bucket temperatures --key '$aws/things/sensor/shadow/update/accepted/2018-07-13/1531514034796' --expression 'S ELECT s.desired.temperature from S3Object s ' --expression-type SQL --input-serialization '{"JSON":{"Type": "document"}}' --output-serialization '{"CSV":{}}' test --profile myprofile works since I specify the one specific key. – Appose

Also see #50721877 – Appose

Possible duplicate of Can S3 Select search multiple objects? – Achondroplasia

Answer 1

Amazon S3 Select operates on only one object.

To query multiple files at once on S3 with a SQL syntax you should use Amazon Athena.

Catechize