Why is gsutil -m cp -R much faster than gsutil cp -R

Question

Hi all I am using GCE and now I'm new to bucket. I have tons of pics on my other bucket. I tried copying it as

sudo gsutil cp -R dir dir

it took me 30 mins still didn't finished the file copy so I tried the suggestion which is to use the -m command

sudo gsutil -m cp -R dir dir

it was a lot faster. But I wonder what makes it faster. and what does -m stand for?

Thanks!

Deoxyribose

Answer 1

From the documentation of gsutil cp:

If you have a large number of files to upload you might want to use the gsutil -m option, to perform a parallel (multi-threaded/multi-processing) copy:

gsutil -m cp -R dir gs://my_bucket'

So the reason is that the -m flag makes the cp command open multiple simultaneous requests to copy the files in parallel, therefore speeding up considerably.

Counterman