Run `apt-get update` before installing other packages with Puppet

Question

I'm trying to create puppet module which automates installation of zend server CE, this is not important here, but steps are as following

1. update /etc/apt/source.list
2. download repos key via wget
3. do apt-get update
4. do apt-get install zend-server-ce-5.2

I have init.pp file

class zendserverce {

# https://github.com/puppetlabs/puppetlabs-stdlib
file_line { 'debian_package':
    path => '/etc/apt/sources.list',
    line => 'deb http://repos.zend.com/zend-server/deb server non-free'
}

exec { "wget http://repos.zend.com/zend.key -O- |apt-key add -":
    path => ["/usr/bin", "/usr/sbin"]
}

exec { "apt-get update":
    command => "/usr/bin/apt-get update",
    onlyif  => "/bin/sh -c '[ ! -f /var/cache/apt/pkgcache.bin ] || /usr/bin/find /etc/apt/* -cnewer /var/cache/apt/pkgcache.bin | /bin/grep . > /dev/null'",
}

package { "zend-server-ce-php-5.2":
    ensure => "latest"
}

}

Seems that puppet runs commands in different order then I need. Is there any way how to for tell him to run in my desired order?

The output of such snippet is

  [0;36mnotice: /Stage[main]/Mc/Package[mc]/ensure: ensure changed 'purged' to 'latest'[0m
  [1;35merr: /Stage[main]/Zendserverce/Package[zend-server-ce-php-5.2]/ensure: change from purged to latest failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install zend-server-ce-php-5.2' returned 100: Reading package lists...
  Building dependency tree...
  Reading state information...
  E: Couldn't find package zend-server-ce-php-5.2 at /tmp/vagrant-puppet/modules 0/zendserverce/manifests/init.pp:28[0m
  [0;36mnotice: /Stage[main]/Zendserverce/Exec[wget http://repos.zend.com/zend.key -O- |apt-key add -]/returns: executed successfully[0m
  [0;36mnotice: /Stage[main]/Zendserverce/File_line[debian_package]/ensure: created[0m
  [0;36mnotice: Finished catalog run in 6.75 seconds[0m

So it says: Couldn't find package zend-server-ce-php-5.2

Can anyone guide me what is wrong?

Demi

maybe this can help docs.puppetlabs.com/guides/language_guide.html#run-stages – Demi

Just pointing out a side issue: your apt-get update will run only once and never execute again while the files in /etc/apt/** arent changed. This can lead to stale update information when installing other packages. You may also want to add this bash check that returns true if the apt-get update was executed more than 1 day ago: [[ $(( $(date +%s) - $(stat -c %Z /var/cache/apt/pkgcache.bin) )) -gt $(( 24 * 60 * 60 )) ]] – Unarmed

A quick note re your apt-key add invocation. Fetching something over HTTP and then passing it to apt-key makes you vulnerable to a variety of security attacks including MITM. It would appear zend don't offer HTTPS access to this key which is a shame. If you can manually verify the key, it would be safer to push it out via another method, perhaps file { ... } – Halpin

Have you tried the puppetlabs-apt module to handle the management of repo and keys? This is not directly related to your question :) – Refute

Answer 1

You need to specify the dependency relationships. The easiest/cleanest approach is to use the require parameter which is available for all resource types.

package { "zend-server-ce-php-5.2":
  ensure  => latest,
  require  => Exec['apt-get update'],
}

etc..

Atp

I read docs.puppetlabs.com/guides/language_guide.html#run-stages and puppet provides stages so you can explicitly specify ordering of execution. Run stage were added in Puppet version 2.6.0, you now have the ability to specify any number of stages which provide another method to control the ordering of resource management in puppet. – Demi

This may be "easiest/cleanest" but is it maintainable? If the Exec needed to change then you have to go back and replace every declaration, the point being this approach isn't referential. – Intuitivism

@BrandonCook Then you can give your exec a new name. exec{update: command => "apt-get update"} then reference it as Exec[update]. – Sezen

If you do this for a number of repos, then you might wind up doing more apt-get updates than you'd like to. Puppet's "stages" are a feature to be careful of, but this is the best use case I know of. – Beef

Answer 2

Since Puppet 2.6.0 a new feature "relationship syntax" was introduced.

An example in Puppet 2.6.0 and above would look like this:

exec { "apt-update":
    command => "/usr/bin/apt-get update"
}

Exec["apt-update"] -> Package <| |>

Every time a package command is executed, the dependency (in our case 'apt-update') will be triggered fist. You can even define longer chains.

Prestige

I actually use stages for this, but I have to admit that this trick is pretty nifty! – Plusch

Thanks, this was super helpful! For other novices like me wondering about that last line: "<|" is collector notation, see docs.puppetlabs.com/puppet/3/reference/lang_collectors.html – Barrage

This seems to run on every puppet update whether a package install is necessary or not. Is there a way to tighten this up so that apt-get update only runs when a package is missing from the system? – Ugly

@Ugly What is your package 'ensure' setting? If it happens also for 'ensure' => 'installed', the relationship syntax could be changed to: Exec["apt-update"] -> Package <| ensure == latest |> But then for every non 'ensure => latest' package, apt-update will not run. – Prestige

Thanks @DrDol! We have ensure set to 'present' but may be moving toward an ensure 'latest' model soon. I think we kind of have a catch 22 issue here. If we bind an apt-get update against "ensure == installed" then it'll run the apt-get update every time before seeing if the package is there. That's the original scenario we're trying to avoid; running the apt-get update all the time. But if we bind apt-get update against "ensure == latest" only then whenever we find a missing package we'll have missed out on doing the apt-get update before doing the install and we might install something old. – Ugly

It's actually not possible to have Puppet sync the exec only if the dependent package resources need syncing. Such things only work by sending events through notify/subscribe. This in turn would only work if the packages do synchronize before the exec. A middle ground may be found in a schedule to have the update run only once per night. - Generally note that the collector syntax has a drawback - it will realize all virtual package resources that may end up being part of the manifest. – Senhauser

Answer 3

I tried previous variant but it doesn't work for me on Ubuntu 10.04

Finaly I prepared the following script, that runs update everytime the repository is older than one week:

exec { 'apt-get update':
    command => "/usr/bin/apt-get update",
    onlyif => "/bin/bash -c 'exit $(( $(( $(date +%s) - $(stat -c %Y /var/lib/apt/lists/$( ls /var/lib/apt/lists/ -tr1|tail -1 )) )) <= 604800 ))'"
}

Hope it helps.

Muskrat

Answer 4

I prefer to put apt-upgrade into a separate stage running before the main stage, so I don't have to hard-wire any dependencies. Check here: https://www.puppet.com/docs/puppet/8/lang_run_stages.html.

A simple example would look like below. It implies you have a separate class for doing the actual apt-update:

stage { "init": before  => Stage["main"] }

class {"apt-update": 
  stage => init, 
  apt_mirror => $apt_mirror 
}

Check my sample LAMPP-box on github to see how the pieces fit together: https://github.com/joerx/vagrant-lampp

Note: be careful with apt-upgrade, as some base boxes break by things like kernel upgrades.

Invest

Awesome. This is a pretty awesome feature I didn't know about. Thanks for sharing. – Campman

Link is dead like a mouse. – Etheleneethelin

Fixed the link, but the relevant part of the manifest is commented out now. You can still see it though. Nothing is static ... – Invest

Answer 5

In Puppet 3 this can be done by realizing virtual resources using resource collectors

# so you don't have to fully qualify paths to binaries
Exec { path => ['/usr/bin'] }    

# virtual resource
@exec { 'sudo apt-get update':
   tag => foo_update
}

# realize resource. filter by arbitrary "foo_update"
# tag and relate it to all Package resources
Exec <| tag == foo_update |> -> Package <| |>

Intuitivism

This doesn't work for me on 3.2.4 unless I change to fully qualified paths. @exec { '/usr/bin/sudo /usr/bin/apt-get update... – Chelseachelsey

Good call I tested this having long ago added Exec { path => ['/usr/bin'] } – Intuitivism

Answer 6

Adding this snippet of voodoo worked for us:

  Apt::Pin <| |> -> Package <| |>
  Apt::Source <| |> -> Package <| |>

This forced an update. YMMV.

Ballflower

Answer 7

Package that needs updating APT lists should require Class['apt::update']

package { "zend-server-ce-php-5.2":
   ensure => "latest",
   require => Class['apt::update']
}

If you're using custom APT source, just ensure correct ordering:

Apt::Source['my_source'] 
-> Class['apt::update']

Dom

Answer 8

You should really be using the apt module to create sources and add keys: https://forge.puppet.com/puppetlabs/apt

If you're using hiera:

apt::sources:
  'artifactory-pro-debs':
    location: 'http://repos.zend.com/zend-server/deb'
    release: 'server
    repos: 'non-free'
    key:
      source: 'http://repos.zend.com/zend.key'

Adams