gsutil Questions
4
Solved
I would like to only get the first level of a fake folder structure on GCS.
If I run e.g.:
gsutil ls 'gs://gcp-public-data-sentinel-2/tiles/'
I get a list like this:
gs://gcp-public-data-senti...
Sessile asked 17/7, 2018 at 10:27
3
Solved
is it possible to compress file or directory of google cloud storage without download it first and re-upload?
I think I need some tools similar like http://googlegenomics.readthedocs.org/en/latest...
Speller asked 23/1, 2016 at 18:57
5
Solved
Is this something that can be done with gsutil?
https://cloud.google.com/storage/docs/gsutil/commands/ls does not seem to mention any sorting functionality - only filtering by a date - which would...
Burgonet asked 18/8, 2017 at 16:3
2
Solved
The gsutil command has options to optimize upload/download speed for large files. For example
GSUtil:parallel_composite_upload_threshold=150M
GSUtil:sliced_object_download_max_components=8
see t...
Copalite asked 19/3, 2019 at 20:16
8
Solved
Is there an option to count the number of files in bucket-folders?
Like:
gsutil ls -count -recursive gs://bucket/folder
Result: 666 files
I just want an total number of files to compare the
...
Pennypennyaliner asked 24/9, 2013 at 15:50
4
Issue - While using the gsutil command to download a file from google bucket with public access from a GCP instance we get the following error
root@openvpn:~# gsutil cp gs://google-artifacts-1/op...
Paunch asked 12/6, 2018 at 9:20
4
I want to create a google bucket if it doesn't exist. Otherwise, I want to reuse the bucket name. How to do it?
Its equivalent of the unix command
mkdir -p dir_name
I used the command but my s...
Tours asked 31/10, 2015 at 13:2
20
I am trying to create an internal app to upload files to google cloud. I don't want each individual user or this app to log in so I'm using a service account. I login into the service account and e...
Rapp asked 15/3, 2018 at 15:2
6
Solved
I am trying to download a folder which is inside my Google Cloud Bucket, I read from google docs gsutil/commands/cp and executed below the line.
gsutil cp -r appengine.googleapis.com gs://my-buc...
Teammate asked 26/3, 2018 at 12:51
10
Solved
I'm currently doing this, but it's VERY slow since I have several terabytes of data in the bucket:
gsutil du -sh gs://my-bucket-1/
And the same for a sub-folder:
gsutil du -sh gs://my-bucket-1/...
Normy asked 9/12, 2014 at 8:13
3
I have created a Google Cloud Platform service account, $GCP_SERVICE_ACCOUNT, with the Storage Admin (roles/storage.admin) role.
Now I wish to restrict this account so that it can only access a spe...
Solo asked 4/12, 2019 at 13:14
3
Solved
I am trying to automate the process of syncing my web assets with Google Cloud Storage. I basically need to copy everything in my development directory up to the cloud. However, I need to ignore th...
Telugu asked 28/3, 2014 at 2:40
6
Solved
Question
Is there a way to grep through the text documents stored in Google Cloud Storage?
Background
I am storing over 10 thousand documents (txt file) on a VM and is using up space. And before...
Attorneyatlaw asked 5/3, 2019 at 2:10
4
Solved
I have a bucket/folder into which a lot for files are coming in every minutes.
How can I read only the new files based on file timestamp.
eg: list all files with timestamp > my_timestamp
Chanson asked 17/5, 2017 at 6:45
3
I followed the Google instructions to set up gsutil. It states that after runnig 'gsutil config' in terminal I should see:
This script will create a boto config file at /.boto
containing your c...
Kymberlykymograph asked 3/11, 2011 at 2:48
3
Solved
I'm trying using gsutil to remove the contents of a Cloud Storage bucket (but not the bucket itself). According to the documentation, the command should be:
gsutil rm gs://bucket/**
However, w...
Photima asked 22/8, 2016 at 8:57
4
I want to know difference between gcloud and gsuitl. Where do we use what? Why certain commands begin with gsutil while others with gcloud?
Evin asked 28/9, 2019 at 10:33
7
Solved
Let's say I've got the following files in a Google Cloud Storage bucket:
file_A1.csv
file_B2.csv
file_C3.csv
Now I want to move a subset of these files, lets say file_A1.csv and file_B2.csv. Cur...
Bison asked 29/4, 2015 at 13:50
2
Solved
After installing and configuring Google Cloud SDK gsutil command can be run by simply typing its name and the argument(-s) using Windows cmd.
Here is the example:
"C:\Program Files (x86)\Google\C...
Subrogate asked 16/4, 2018 at 21:10
7
Solved
I am using Google Cloud Storage and Google Compute Engine and am trying to upload to my bucket from my VM. My bucket has read/write permissions and so does anyone that has access to it. They are la...
Nerty asked 19/2, 2015 at 16:50
8
According to Chrome dev tools, my requests to get my html partials have the origin header https://site-name-here.com and request header GET.
I have the following JSON file set to my bucket:
[
{
...
Rhigolene asked 16/3, 2016 at 21:15
2
Solved
I am looking for a way to pipe the data into gsutil cp:
echo '<html>foobar</html>' | gsutil cp --stdin gs://my-bucket/foo
the -I option is for reading filepaths, not the data..gsutil...
Nitrile asked 24/1, 2020 at 21:29
5
I am trying to follow this simple Dataflow example from google cloud site.
I have successfully installed the dataflow pipeline plugin and gcloud SDK (as well as Python 2.7). I have also set up a ...
Armillda asked 19/3, 2016 at 10:16
7
Solved
I am trying to create a bucket using gsutil mb command:
gsutil mb -c DRA -l US-CENTRAL1 gs://some-bucket-to-my-gs
But I am getting this error message:
Creating gs://some-bucket-to-my-gs/...
Bad...
Lougheed asked 16/6, 2014 at 17:17
4
I would like to copy a list of files to a bucket while keeping the directory-structure.
test.txt:
a/b/1.jpg
a/c/23.jpg
a/d/145.jpg
gsutil command:
cat file.txt | gsutil -m cp -I 'gs://my-buck...
Dulin asked 2/12, 2015 at 15:15
1 Next >
© 2022 - 2024 — McMap. All rights reserved.