Chef does non sequential recipe execution

Question

I followed tutorial http://gettingstartedwithchef.com/, chapter 1.

My run list is

  "run_list": [ "recipe[apt]", "recipe[phpap]" ]

My default recipe of phpap cookbook

include_recipe "apache2"
include_recipe "build-essential"
include_recipe "openssl"
include_recipe "mysql::client"
include_recipe "mysql::server"
include_recipe "php"
include_recipe "php::module_mysql"
include_recipe "apache2::mod_php5"    
include_recipe "mysql::ruby"

Dependencies of my cookbook

depends "apache2"
depends "mysql"
depends "php"
depends "database"

My repo has following downloaded cookbooks

apache2  aws              database  openssl  phpap    xml
apt      build-essential         mysql  php      postgresql  xfs

I use chef-solo. My host has outdated apt repo info. Old apt repo should not be a problem because the first recipe in my run list updates it. But chef ignores apt recipe and starts from mysql one.

See log

dan@mywp3:~/chef-repo$ sudo chef-solo -c solo.rb -j web.json 
Starting Chef Client, version 11.6.2
Compiling Cookbooks...
[2013-10-27T00:59:28+04:00] WARN: Cloning resource attributes for service[apache2] from prior resource (CHEF-3694)
[2013-10-27T00:59:28+04:00] WARN: Previous service[apache2]: /home/dan/chef-repo/cookbooks/apache2/recipes/default.rb:24:in `from_file'
[2013-10-27T00:59:28+04:00] WARN: Current  service[apache2]: /home/dan/chef-repo/cookbooks/apache2/recipes/default.rb:210:in `from_file'
[2013-10-27T00:59:28+04:00] WARN: Cloning resource attributes for directory[/var/cache/local/preseeding] from prior resource (CHEF-3694)
[2013-10-27T00:59:28+04:00] WARN: Previous directory[/var/cache/local/preseeding]: /home/dan/chef-repo/cookbooks/apt/recipes/default.rb:76:in `block in from_file'
[2013-10-27T00:59:28+04:00] WARN: Current  directory[/var/cache/local/preseeding]: /home/dan/chef-repo/cookbooks/mysql/recipes/server.rb:44:in `from_file'
[2013-10-27T00:59:28+04:00] WARN: Cloning resource attributes for directory[/var/lib/mysql] from prior resource (CHEF-3694)
[2013-10-27T00:59:28+04:00] WARN: Previous directory[/var/lib/mysql]: /home/dan/chef-repo/cookbooks/mysql/recipes/server.rb:117:in `block in from_file'
[2013-10-27T00:59:28+04:00] WARN: Current  directory[/var/lib/mysql]: /home/dan/chef-repo/cookbooks/mysql/recipes/server.rb:117:in `block in from_file'
[2013-10-27T00:59:28+04:00] WARN: Cloning resource attributes for template[/etc/mysql/my.cnf] from prior resource (CHEF-3694)
[2013-10-27T00:59:28+04:00] WARN: Previous template[/etc/mysql/my.cnf]: /home/dan/chef-repo/cookbooks/mysql/recipes/server.rb:134:in `from_file'
[2013-10-27T00:59:28+04:00] WARN: Current  template[/etc/mysql/my.cnf]: /home/dan/chef-repo/cookbooks/mysql/recipes/server.rb:194:in `from_file'
Recipe: mysql::client
  * package[mysql-client] action install
================================================================================
Error executing action `install` on resource 'package[mysql-client]'
================================================================================


Chef::Exceptions::Exec
----------------------
apt-get -q -y install mysql-client=5.5.32-0ubuntu0.12.04.1 returned 100, expected 0


Cookbook Trace:
---------------
/home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:44:in `block in from_file'
/home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:43:in `each'
/home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:43:in `from_file'
/home/dan/chef-repo/cookbooks/phpap/recipes/default.rb:20:in `from_file'


Resource Declaration:
---------------------
# In /home/dan/chef-repo/cookbooks/mysql/recipes/client.rb

 46:   package name
 47: end



Compiled Resource:
------------------
# Declared in /home/dan/chef-repo/cookbooks/mysql/recipes/client.rb:46:in `block in from_file'

package("mysql-client") do
  action :install
  retries 0
  retry_delay 2
  package_name "mysql-client"
  version "5.5.32-0ubuntu0.12.04.1"
  cookbook_name :mysql
  recipe_name "client"
end




================================================================================
Recipe Compile Error in /home/dan/chef-repo/cookbooks/phpap/recipes/default.rb
================================================================================


Chef::Exceptions::Exec
----------------------
package[mysql-client] (mysql::client line 46) had an error: Chef::Exceptions::Exec: apt-get -q -y install mysql-client=5.5.32-0ubuntu0.12.04.1 returned 100, expected 0


Cookbook Trace:
---------------
  /home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:44:in `block in from_file'
  /home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:43:in `each'
  /home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:43:in `from_file'
  /home/dan/chef-repo/cookbooks/phpap/recipes/default.rb:20:in `from_file'


Relevant File Content:
----------------------
/home/dan/chef-repo/cookbooks/mysql/recipes/ruby.rb:

 37:    when 'rhel'
 38:      resources('yum_key[RPM-GPG-KEY-percona]').run_action(:add)
 39:      resources('yum_repository[percona]').run_action(:add)
 40:    end
 41:  end
 42:  
 43:  node['mysql']['client']['packages'].each do |name|
 44>>   resources("package[#{name}]").run_action(:install)
 45:  end
 46:  
 47:  chef_gem 'mysql'
 48:  


[2013-10-27T00:59:30+04:00] ERROR: Running exception handlers
[2013-10-27T00:59:30+04:00] ERROR: Exception handlers complete
[2013-10-27T00:59:30+04:00] FATAL: Stacktrace dumped to /home/dan/chef-solo/chef-stacktrace.out
Chef Client failed. 0 resources updated
[2013-10-27T00:59:30+04:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

The host runs ubuntu 12.04. I tried to put my own recipes before phpap in the run list but without any success.

I can workaround this with manual placing apt-get update in bash script right before chef-solo.

Is it mysql cookbook so bad? As far as I know chef ALWAYS follows run-list strait forward.

Beaird

did you check if this file exists: /var/lib/apt/periodic/update-success-stamp? the command apt-get update won't be executed if this file exists. – Hamlani

apt-get is just an example. I tried my own handmade recipes doing just printing hello world with the same result. – Beaird

Answer 1

In general chef will always honor the run list. However sometimes you might want to run a resource before any others (like to configure a package manager, apt/yum/etc). What seems to be happening in the mysql ruby recipe is that the recipe is explicitly running some package installs at compile time (before chef has begun to execute resources). They fail as the apt recipe hasn't been run yet (none have).

There's a good opscode post explaining compile/execute phases here.

The offending code in mysql::ruby is

node['mysql']['client']['packages'].each do |name|
  resources("package[#{name}]").run_action(:install)
end

The run_action(:install) is telling chef to run this now (at compile time).

Indestructible

I had a link to a wiki article on the documentation of the difference between compile and execute phases of Chef recipes, but now it is 404'ing. I will try to post it if I ever get a working url... – Milford

Damnedest thing: the link doesn't work without the jsessionid param... (taken straight from a google search result) wiki.opscode.com/display/chef/… – Milford

That link was already in the post, put in a line break to make it a little more obvious. Thanks, though. – Indestructible

That link to wiki.opscode.com is now dead. This blog has a reasonable explanation of the two phases of Chef execution. – Fully

Answer 2

I got phpap recipe without failure on bare ubuntu 12.04 (without any manual pre update) with following fix.

I removed recipes "build-essential" and "openssl" from phpap one. After that change chef executes recipes as expected.

But weird behavior left unexplained.

Beaird

Answer 3

vagrant destroy -f && vagrant up // destroy everything and start over => OK

Jacie