Querying struct fields from AWS Athena/Presto
Asked Answered
E

1

6

I'll make a simplified example for this site, but basically I'm trying to write an Athena query (of data loaded by Glue crawler with intent to use in Quicksight) which will allow me to expand a struct inside of a select statement.

In my example, lets say I have a table my_table which is similar to:

id string,
scores struct<prediction:double,score:int>

But since I do not know the exact structure at query time of the scores column, I would like to expand it in the scope of a query.

I have tried many combinations of things, but only using the exact name of the nested field inside of the struct seems to yield results, i.e.

select results.id, results.scores.score, results.scores.prediction from my_table results

I would like it if something like select results.id, results.scores.* from my_table results worked, but unfortunately it doesn't.

Eccentric answered 11/9, 2019 at 15:30 Comment(0)
Q
6

I would like it if something like select results.id, results.scores.* from my_table results worked, but unfortunately it doesn't.

What you're asking for is called "generalized all fields reference". It is available in Presto since Presto 323. You can download the latest Presto release from https://trino.io/download.html.

If you are using Athena, then unfortunately, Athena is currently based on Presto .172 (released 3 years ago), so this functionality is not available there. Until Athena is upgraded, there is no workaround you can apply on SQL level. You need to compensate this within application reading the data from Athena. Or, you can deploy recent Presto version on AWS easily.

Quinte answered 11/9, 2019 at 19:58 Comment(1)
Please elaborate on the syntax for the "generalized all fields reference", with an example.Roby

© 2022 - 2024 — McMap. All rights reserved.