gcloud sql import csv fails with permissions error when import file contains wildcard

Question

According to the Google Cloud Platform SQL docs, I should be able to both export to and import from sharded files in a GCS bucket by putting a * in the filename.

If I import a single file, it works fine:

gcloud sql import csv sql-instance-name gs://gcsbucketname/data/led/led_finance_view_000000000000.csv --project=project-name --database=finance --table=Import_Test -q
Importing data into Cloud SQL instance...done.
Imported data from [gs://gcsbucketname/data/led/led_finance_view_000000000000.csv] into [https://www.googleapis.com/sql/v1beta4/projects/project-name/instances/sql-instance-name].

But if I import a sharded file, it throws a permissions error:

gcloud sql import csv sql-instance-name gs://gcsbucketname/data/led/led_finance_view_*.csv --project=project-name --database=finance --table=Import_Test -q
ERROR: (gcloud.sql.import.csv) HTTPError 403: The service account does not have the required permissions for the bucket.

I can confirm that these commands are running under the same user, using the same bucket & SQL instance.

I think the '403 service account permissions' error is probably a bug in GCP and therefore a red herring - but why won't it let me import the sharded file?

Kerrykersey

Can you add a link to the docs where it specifies this? – Alidis

Answer 1

Currently, CloudSQL doesn't incorporate the possibility of importing several CSV files at once using wildcards. I have opened a public feature request for this. You can star to make it gain visibility.

Meanwhile, as a workaround, you can either have a script to run the import command once for each file, or join the CSV files before importing.

Coupling

Answer 2

You have to get the user id (of creation) of your current database instance, in order to get that user id, type something like this in your terminal:

Command: gcloud sql instances describe short [name/db instance]

After that , go to the gcs project -> open your bucket -> add member permitions and you have to add the user id that you got in first command, add role for Storage Admin for that user id.

and that´s all.

Excitable

gcloud sql instances describe short is a command that does not exist – Ujiji