AWS S3 SELECT WebUI error: "Quoted record delimiter found in the file"

Question

I am trying to the Select From for CSV in AWS S3 SELECT Web UI, but I get the following error msg:

Quoted record delimiter found in the file. To allow quoted record delimiters, please set AllowQuotedRecordDelimiter to 'TRUE'.

But there is no option to set AllowQuotedRecordDelimiter to TRUE

Please advise

Beekeeper

I think you need python boto module for this to work. Add the parameter to s3.select_object_content. For e.g. InputSerialization = {'CSV': {"FileHeaderInfo": "Use", 'AllowQuotedRecordDelimiter': True,}}, as https://mcmap.net/q/1012707/-s3-select-csv-headers – Cyrillus

It's a pity that this is not available in the UI. Shouldn't have to run a script just to do an S3 Select – Premature

1 year later - AllowQuotedRecordDelimiter SHOULD default to TRUE in the UI but still doesn't. – Lactoprotein

and another year passed :) – Furtherance

yes, this should have been an easy fix for AWS team and quick win for the UI. – Splore

This is still an issue. I just submitted a feature request for S3 SELECT to expose all input and output settings using the "Feedback" button in the bottom left-hand corner of the AWS Console. I recommend everyone do the same. – Reverence

Answer 1

Try adding a " in the custom field under the CSV delimiter section, see picture bellow (This did the trick for me):

Xerophilous

Answer 2

While writing to_csv

set sep = ' " '

pd.to_csv('example_df.csv',sep='"')

It should work.

Internalize