How I can migrate a Persistence Disk from one project to another, in Compute Engine?

Question

I would like to know how I can migrate a Persistence Disk (Google Compute Engine) from one project to another? If it's posible.

Newsboy

Answer 1

There is no easy way to do this on GCE, the only way is to go through the process of creating a custom image, exporting it to a cloud storage bucket (that both projects have permissions on), and then creating a new instance using the new custom image in the new project.

Follow these instructions: https://developers.google.com/compute/docs/images#installinganimage

Lutes

I knew there is not easy way, but at less there is a way. I hope, they change this. Thanks! – Newsboy

I'm dockerizing all my things, I need to be able to move my stuff between projects in routine way. – Volant

There is a way, using disk images and the URL of the disk image, so long as you have EDIT permissions on both projects. This answer is somewhat out-dated. See Amos Folarin's answer below. – Measureless

Answer 2

If you want to copy (non-root) persistent disk, do the steps below:

1. Go to Google Developer Console and detach <disk-name> disk from the machine.
2. Use gcloud command-line utility to switch to the old project:
   • gcloud config set project <old-project>
3. Create an image of the <disk-name> disk:
   • gcloud compute images create <image-name> --source-disk=<disk-name> --source-disk-zone=<zone>
4. The output of the above command will give you a fully qualified link to the image (in the form https://www.googleapis.com/compute/v1/projects/<project-name>/global/images/<image-name>). Alternatively, run: gcloud compute images list --no-standard-images --uri | grep <image-name> to find the link.
5. Switch to the new project:
   • gcloud config set project <new-project>
6. Create a new disk from the image:
   • gcloud compute disks create <new-disk-name> --image=<link-to-the-image> --zone=<zone>

Mccombs

For command line addicts, step 4 can be replaced by gcloud compute images list --no-standard-images --uri, which gives you all the link of your non-standard images. – Beeves

Those who don't want to detach the disk from running node can take snapshot of the disk and create a disk from that snapshot. Use that disk to create image as suggested above. – Relational

Answer 3

In case anyone else lands here from a google search, today you can create a disk in one project directly from a snapshot in another project, without creating any temporary physical disks or images.

1. Create snapshot: gcloud compute disks snapshot your-disk-name --snapshot-names=your-snapshot-name --project=source-project --zone=source-zone

2. Describe the snapshot and find the selfLink: gcloud compute snapshots describe your-snapshot-name --project=source-project

3. Use the selfLink instead of a snapshot name when creating a disk in the new project: gcloud compute disks create your-new-disk-name --project=destination-project --zone=destination-zone --source-snapshot=https://www.googleapis.com/compute/v1/projects/source-project/global/snapshots/your-snapshot-name

Truncated

Answer 4

As above Mike Lutz's answer, except gcutil is now deprecated, but the 2nd command can also be done with gcloud compute instance are:

1) create your image from you PD (NB! read first! https://cloud.google.com/compute/docs/images#creating_an_image_from_a_root_persistent_disk)

$ gcloud compute images create [example-image] --source-disk [example-disk] --source-disk-zone ZONE --project="old-project-name"

2) Instantiate the image in the new project (goes without saying but you must have access to both projects)

$ gcloud compute instances create [example-instance-1] --project=[new-project-name] --image="https://www.googleapis.com/compute/v1/projects/[old-project-name]/global/images/[image-name]" --boot-disk-size [XXXGB] --machine-type=[machine-type] --network="default" --zone=[datacenter-zone] 

• you can see the URL of your image in the Images tab under "Equivalent REST"

For additional instance config options see: https://cloud.google.com/sdk/gcloud/reference/compute/instances/create

Volant

The instance create command should be with --network="default" instead of network="default", but other than that this worked for me :) – Harmonium

Answer 5

If you can create an image out of it. Then you can do something like this and it will copy the image to a disk in you new project:

{gcloud compute instances create my-instance-1 --project=[new-project-id] --image="https://www.googleapis.com/compute/v1/projects/[old-project-id]/global/images/[image-name]"}

Kadi

Answer 6

Adding on Amos' answer above, in order to avoid having to detach a pd from its instance in order to make an image out of it, I used the following sequence:

In the source project:

1. Create a SNAPSHOT from the desired boot disk to be cloned.
2. Create a new persistent disk from that snapshot.
3. Create an image from the new pd.

In the target project:

1. Create a VM instance using the url of the newly created image.

When you're done you can delete the created disk and image in the source project.

Guarantor

Answer 7

While the following won't directly migrate a Persistence Disk, it will let you make a copy of one on a different project with only two commands.

If you combine process in the document IanGSY references for turning the disk into an "image" I.E:

gcloud compute images create example-image --source-disk example-disk --source-disk-zone ZONE

(above command only works if the drive isn't live (attached), if you have to drop the VM make sure you don't let it delete the drive - make sure "Delete boot disk when instance is deleted" isn't checked if you use the web UI!)

And use what is hinted at by Jens, (which is spelled out in the answer to SO: How to share google compute engine images across projects?) I.E:

gcutil --service_version="v1" --project=<ID-of-your-project> addinstance <name-you-for-instance> --machine_type=<type-of-machine> --network="default" --external_ip_address="ephemeral" --zone=<datacenter-zone> --image="https://www.googleapis.com/compute/v1/projects/<ID-of-image-owners-project>/global/images/<image-id>" --persistent_boot_disk="true"

You will end up with a copy of the given disk on the other project.

Whitehead

Answer 8

I had an awful time getting data on a disk from one of my projects to another. Tried my best to follow all the directions here and from Google but couldn't get a brand new Disk created without creating an Instance first. Here was what I ended up doing... (highly influenced from many other answers)

In original project:

• Go to IAM and look for your email address. In Role(s) add the role: Compute Engine => Compute Image User (this assumes you're the owner of both projects. I'm also not 100% positive this was a necessary step in the end)
• Create a new Image. Set the "Source" to be "Disk". Then select your disk.

In destination project:

• Open Google Cloud Shell (button on top right of GCE console)
• Run this command to create a new INSTANCE from the disk: gcloud compute instances create NAMEFORNEWINSTANCE --image IMAGENAMEYOUCHOSE --image-project ORIGINALPROJECTNAME (Yes, we're creating a new instance even though you really only want to create a new disk. I couldn't get it to work just creating a new disk.)
• Find the Instance that was just created. Stop it. Edit it. Uncheck the box for "Delete boot disk when instance is deleted". Save Changes. Delete the Instance. Don't check the box to delete the boot disk.
• Go to the Disks panel. You should see a disk named the same as whatever you picked for NAMEFORNEWINSTANCE

Attach Disk: (not exactly the same topic but putting the steps here for anyone who needs to know how to do this)

• In Google Cloud Shell run: gcloud compute instances attach-disk NAMEOFINSTANCETOATTACHTO --disk=NAMEOFDISK (Again, NAMEOFDISK should be the same as what you picked for NAMEFORNEWINSTANCE) You should see in the Disks panel that it's now attached to your new instance.
• SSH into Instance you attached to
• Create a new directory to mount this disk to: sudo mkdir /DIRECTORYNAME
• Run lsblk to see a list of disks, their names, sizes, and mountpoints. Try to find your attached disk in there. The size should match what you see in the Disks panel and should not be mounted to anything
• Mount the disk to your directory using: sudo mount /dev/NAME_FOUND_IN_STEP_ABOVE /DIRECTORY

Marella

Answer 9

There is the official guide that only requires 3 steps to achieve the goal as snapshots can be accessed cross projects: https://cloud.google.com/compute/docs/disks/create-snapshots#sharing_snapshots

gcloud compute disks snapshot disk-1 --project project-a --snapshot-names snapshot-1
gcloud compute disks create disk-2  --project project-b --source-snapshot projects/project-a/global/snapshots/snapshot-1
gcloud compute instances attach-disk instance-1 --project project-b --disk disk-2

Pauperize

@thư-sinh, I gave that answer first, though. Lol. – Pauperize

Answer 10

Adding my answer because everything seems to be dated. GCP allows images to be moved between projects, and the entire solution revolves around this.

I have 2 GCP projects A and B. The persistent disk in A needs to be transferred to B.

 1. Create an image from your persistent disk in A.
 2. In B, create a new image and select the source as the image from A.
 3. In B, create a disk from the image (from 2)
 4. (Cleanup) Delete the images in both A and B.

All this can be done via the UI (15 Jan 2021).

Gigi

Answer 11

Its nearly 2023 and the answer is much simpler. Here's what I did to clone a regional disk from one project into another:

DISK_NAME=source-disk
SOURCE_PROJECT=A
SOURCE_REGION=europe-west1
TARGET_PROJECT=B
TARGET_REGION=europe-north1
TARGET_REPLICA_ZONES=europe-north1-a,europe-north1-c
gcloud compute disks create $DISK_NAME \                                                                                                                                                                       
  --region=europe-north1 \
  --source-disk=https://www.googleapis.com/compute/v1/projects/$SOURCE_PROJECT/regions/$SOURCE_REGION/disks/$DISK_NAME \
  --replica-zones=$TARGET_REPLICA_ZONES \
  --project=$TARGET_PROJECT

The steps are similar if you want to clone a zonal disk: https://cloud.google.com/compute/docs/disks/create-disk-from-source#clone

Yadirayaeger

Answer 12

Don't use the process of creating an image file in a Cloud Storage bucket and sharing it. It is not very robust (yet). Amos' method of creating an image in the source project and creating an instance from that image in the destination project is much cleaner and worked great for me.

Do pay some attention to the command line options, as the gcloud utility is in a bit of flux.

Stoned

Answer 13

If the data is not too big (e.g. under 1 TB(?)), it might be easier to just copy it directly using two VMs, each with the attached disk:

• On target VM, generate a temporary ssh key: ssh-keygen -t rsa
• On source VM, use console to add content of the generated public key (from target's ~/.ssh/id_rsa.pub). You can also just add it as a new line to target's ~/.ssh/authorized_keys)

• Use rsync or scp to copy the data.

• If your data requires sudo, you would need an extra step - enable sudo rsync:

  • On source, use sudo visudo and add username ALL=NOPASSWD:/usr/bin/rsync ( change username, and use which rsync just in case it has a different path)
  • Use rsync -av --rsync-path="sudo rsync" -e='ssh -o "UserKnownHostsFile /dev/null" -i ~username/.ssh/id_rsa' [email protected]:/source/path/ /target/path (update username twice and use proper source.ip.address)

Incarnation

Answer 14

Using the GUI

Inside project1:

1) grant access images by going to https://console.cloud.google.com/iam-admin/iam and click Add then enter the email associated with project2, and select role as Compute Engine > Compute Image User (might also need Compute Storage Admin role)

2) create an image from disk https://console.cloud.google.com/compute/imagesAdd

Inside project2:

3) create a new instance and under 'boot disk' click change https://console.cloud.google.com/compute/instancesAdd

4) select the 'custom images' tab and under 'select images from' select project1

5) select the shared disk name listed

6) continue with the new instance setup

Mechelle

Answer 15

This is quite easy but google documentation does not explain it very well.

Create snapshot in source project with console or gcloud command:

gcloud compute disks snapshot DISK_NAME --project SOURCE-PROJECT --snapshot-names SNAPSHOT-NAME --zone SOURCE_DISK_ZONE

then create disk with this snapshot in destination project:

gcloud compute disks create NEW-DISK --zone DEST_ZONE --project DEST-PROJECT --source-snapshot projects/SOURCE-PROJECT/global/snapshots/SNAPSHOT-NAME

Airplane

Answer 16

The method, described by Amos above, uses the command "gcloud compute instances create" to solve the task requested by the OP. When he makes use of the "--image" flag, he provides the google url (which is long, and I have to look it up and check it for syntax each time).

The same "gcloud compute instances create" command also takes an "--image-project" flag, in which the task requested by the OP gets solved in the same way, except, it accepts your image and project names so you don't have to remember or lookup the longer uri.

The "--image" and "--image-project" flags are also accepted by the command: "gcloud compute disks create", so you can migrate the original project's image of the original project's persistent disk, over to a new persistent disk in a new project, without having to create a running (i.e. billable) instance.

Either method requires that an image of the disk to be migrated is created in the original project before creation of either instance or disk in new project.

I just migrated a disk from my development project to a production project, and my session looked as follows:

1. in my original project, I created an image of the disk I wanted to migrate (first it had to be detached from any running instances) Someone else might have a different disk zone than the one I was using, so modify accordingly:

   gcloud config set project originalproject

   gcloud compute images create imageofmydisk --description "image of my disk, ready for deployment" --family "grsec-4.6.5" --source-disk mydisk --source-disk-zone us-central1-f

2. switching to my new project, I create a new disk, based on my newly created image of my disk in my old project:

   gcloud config set project newproject

   gcloud compute disks create mynewdisk --description "description of mynewdisk" --type pd-ssd --image imageofmydisk --image-project originalproject

Thus I accomplish the task requested by OP, without having to memorize, or look up each time, the full url to the image.

Photomural