How to attach EBS volume with Packer during build time?

Question

I'm not sure if this is the right way to go about it but what I'm looking to do is shorten the build time of an image by Packer. One of the steps required is to copy some 10 GB of data and make it part of the image.

Currently I'm using a shell provisioner to SFTP this in to the image. This takes a long time. Instead I would like to have this data already in an EBS volume which can be attached at build time. Since it would be mounted as a block device in Amazon's own network the transfer would be much faster than SFTP.

I tried searching around for such a method in Packer but there's not much out there. I'm going to try and see if I can use the ami_block_device_mappings optional parameter in the amazon-ebs builder to attach the EBS volume.

If there is a better method I'm not aware of please let me know.

Piercing

Answer 1

So it looks like the shell provisioner is the way to go..for now. The shell provisioner seems like the catch all for things like this :). It would be good to have a parameter in the amazon-ebs builder at some point though. Something like "attach_ebs_volume" for instance. Anyway, here's how I did it:

1. Add AWS info (ebs_volume, instance_id etc..) to the variables secion of the template. I do pull the access and secret key from ENV so it's not spelled out in the template but the rest i just define in the template for now.
2. use "environment_vars" parameter in the provisioners block in the template to pass the variables to the build machine.
3. add {{.Vars}} to the "execute_command" parameter so the vars are defined prior to the script run.
4. install aws cli (part of the script being passed.)
5. grab the instance_id of the build machine from meta_data via wget and assign to an env var.
6. issue the aws cli command to attach the volume and use the env vars passed from the template to fill in the command options as well as the instance_id from the wget command.
7. mount as normal (have it sleep for a few seconds before mounting, it takes a little time to attach. Otherwise it won't see it and the mount will fail.)

Done.

If anyone has a more elegant way... please share.

Thanks.

Piercing

Hi, any chance you could post a full example of this solution? – Allfired

Answer 2

I think the ami_block_device_mappings is what you need.

I'v also found this example helpful :

http://grahamc.com/blog/packer-ami-device-volume-types/

Andrews

Thanks. That does clarify things for me a bit. I still don't get how I can specify the ebs volume to be mounted though. I think this allows you to create and attach the volume but how do you attach an existing volume? I don't see an option to specify a volume id. Something like this: { ` "device_name": "/dev/sdg",` ` "volume_id": "vol-xxxxx"` ` }` – Piercing

I don't see such option but you could create an AMI with an attached EBS volume if you have a snapshot of that volume. In that case use the option "snapshot_id" . – Andrews