How to use cloud-init with a debian-based image on Google Cloud?

Question

Following terraform best practice for bootstrapping instances, I'm working on a cloud-init config in order to bootstrap my instance. My only need is to install a specific package.

My terraform config looks like this:

resource "google_compute_instance" "bastion" {
  name         = "my-first-instance"
  machine_type = "n1-standard-1"
  zone         = "europe-west1-b"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
    }
  }

  network_interface {
    network = "default"

    access_config {
      // Ephemeral IP
    }
  }

  metadata = {
    ssh-keys = "eugene:${file("/Users/eugene/.ssh/id_rsa.pub")}"
    user-data = file("my_cloud_init.conf")
  }
}

Following example for installing packages from cloud-init docs, here's the contents of my_cloud_init.conf:

#cloud-config

packages:
 - kubectl

After running terraform plan -out myplan and terraform apply myplan, I ssh onto the node only to find kubectl not available. Moreover, there's no evidence that cloud-init was run or that it exists on the node:

$ which -a cloud-init
$ cat /var/log/cloud-init
cat: /var/log/cloud-init: No such file or directory

Looking for clues about usage of cloud-init with Google Cloud Compute instances wasn't fruitful:

• "Google Cloud Engine" page from cloud-init docs suggests settings user-data to a cloud-init config should be enough,
• I see a cloud-init tutorial, but it's for Container Optimized OS,
• there are some clues about cloud-init on other images, but nothing indicates cloud-init is available on debian-cloud/debian-9,
• there's "Running startup scripts", but it has no mention of cloud-init.

I don't mind using another image, as long as it's Debian or Ubuntu and I don't have to make an image template myself.

How to use cloud-init with a debian-based image on Google Cloud? What am I missing?

Puttyroot

I don't have to work on kubectl, but also need docs on cloud-init for GCP and here's what I found: A request to add it to docs, then the open MR in official docs on hashicorp website, and that's how I found the incomplete documentation for that. Unfortunately it's still an AWS-specific doc, but just set base64 and gziping to false. Helpful? – Metamathematics

Answer 1

cloud-init is installed on the latest (at the moment of writing) Ubuntu 18.04 LTS (ubuntu-1804-bionic-v20191002) image :

<my_user>@instance-1:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"

<my_user>@instance-1:~$ which cloud-init
/usr/bin/cloud-init

You should replace debian-cloud/debian-9 with ubuntu-os-cloud/ubuntu-1804-bionic-v20191002.

Twospot

Thank you, @norbjd! Did you know this from experience, or were you able to look it up just now? The reason I am asking is because googling failed me, and I want to understand how to properly search for it next time. – Puttyroot

I just checked by starting an Ubuntu instance :) Sadly, I don't know how to find what packages are installed on a specific public image without trying. – Twospot

Answer 2

To complement answer from @norbjd, posting this for completeness:

• in order to install kubectl package that I planned initially, I had to write this cloud-init config:

    #cloud-config
  
    bootcmd:
     - curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
  
    apt:
      sources:
        kubernetes.list:
          source: "deb http://apt.kubernetes.io/ kubernetes-xenial main"
  
    packages:
     - kubectl
  

• if you need to know which images contain which packages, these resources may be helpful:

  • Ubuntu:

    • Ubuntu Cloud Images (RELEASED)

    • Actual contents (example):

      • https://cloud-images.ubuntu.com/releases/releases/23.04/release-20230729/ubuntu-23.04-server-cloudimg-amd64.manifest

  • Debian:

    • List of differences between standard and cloud systems

    • Actual contents:

      • https://salsa.debian.org/cloud-team/debian-cloud-images/-/blob/master/config_space/sid/package_config/EXTRAS
      • https://salsa.debian.org/cloud-team/debian-cloud-images/-/blob/master/config_space/sid/package_config/CLOUD

Puttyroot