Multistage deployment with ansible

Question

What approach would you advise to organize multistage deployment with Ansible in case you have different variables for stages?

The main idea is defining group variables for different stages.

There are two articles:

• http://rosstuck.com/multistage-environments-with-ansible/
• http://toja.io/using-host-and-group-vars-files-in-ansible/

I'd like to get more examples about organizing playbooks, variables, and hosts, and understand advantages and disadvantages of your approach.

Burweed

Can you give an example of what you mean? – Endstopped

Answer 1

Recently, I used the approach I had already mentioned in the question, and it occurred to be one of the most convenient to my mind.

It is taken from Organizing Group Vars Files in Ansible article, but altered a little, because, unfortunately, title of the article does not reflect it's real value and purpose and names of playbooks are confusing as well. In fact, it took a considerable time to realize that it is about Multistage deployment with Ansible.

Your layout of directories should be like that:

production/
├── group_vars
│   └── server.yml
└── inventory
staging/
├── group_vars
│   └── server.yml
└── inventory
deploy.yml

And usage is extremely simple:

ansible-playbook -i staging deploy.yml

Where deploy.yml is the name of your playbook.

Ansible-playbook when provided a directory as the inventory, will search by default a file named inventory so no need to specify -i production/inventory, only -i production will work just fine.

And the benefits are:

• You don't have to mantain some unnecessary groups like [production:children]

• You don't need to keep confusing groups and files like like group_vars/production.yml

• All vars and hosts are in separate directories, therefore it is easy to keep them different and history of changes is clear. You may even split it in separate repositories if you want

You may also keep secrets for production in repository using ansible-vault, in other words, store all your vital variables encrypted

Burweed

What about overlapping groups from different environment. Say, webservers which might consists of hosts from all three environments. How do you handle it? And to add complexity, all webservers from staging environment. – Monandrous

@Monandrous that seems to be an architecture problem. Do you really have a host which supports 2 different environments simultaneously? Keeping multiple intersected configurations installed might result in trouble. Of course, you can use version management systems or add some conditions to Ansible and so on, but I'd recommend to add another virtual layer – Burweed

@Monandrous Can you give detailed description of your problem? In chat if you don't mind – Burweed

I'm a bit confused here about where non-environment-specific group_vars would go in this setup. Would Ansible load both the group_vars file in the environment directory and also one that's outside of it? – Potbelly

@Potbelly doc says "The group_vars/ and host_vars/ directories can exist in the playbook directory OR the inventory directory. If both paths exist, variables in the playbook directory will override variables set in the inventory directory" – Burweed

how do you share variables across environments? for example..maybe all our ports for a webserver are same across environments. or perhaps the paths to where you write logs for these hosts are same.... – Hengelo

Answer 2

In case of complex inventory structures, when maintaining groups of groups is not the best option ( http://docs.ansible.com/ansible/intro_inventory.html#groups-of-groups-and-group-variables ) the following trick can be used:

Production Inventory File:

# production inventory 
[loadbalancers]
lb01
lb02
lb03

[webservers]
ws01
ws02
ws03

[all:vars]
inventory_vars=prod.config.yml

Development Inventory File:

# development inventory 
[loadbalancers]
test-lb01
test-lb02
test-lb03

[webservers]
test-ws01
test-ws02
test-ws03

[all:vars]
inventory_vars=development.config.yml

Now in playbook itself, include the following task before loading roles:

- hosts: all

  pre_tasks:

    - name: Load inventory specific variables
      include_vars: "{{ inventory_vars }}"

To prevent accidental execution of playbook on production environment, prod.config.yml cab be encrypted with ansible-vault

Mauritamauritania

I am going to reaccept answer. Yours was very helpful for me, but another one (my own) matches my needs 100%, although it does not have any upvotes yet. In fact I could've dig a bit deeper in links I provided myself in question to find out desired result. But the article I referenced to it is a bit vague, that's why I composed another answer with broader explanations – Burweed

Answer 3

Currently I'm using the following structure:

hosts/development
hosts/production
hosts/group_vars/development/service1.yml
hosts/group_vars/development/service2.yml
hosts/group_vars/production/service1.yml
hosts/group_vars/production/service2.yml
hosts/group_vars/production/service3.yml
hosts/host_vars/dev1.yml
hosts/host_vars/prod1/something.yml
hosts/host_vars/prod1/something_else.yml

The inventories could look like this:

# hosts/development

dev1 ansible_ssh_host=dev1.example.com
dev2 ansible_ssh_host=dev2.example.com

[development]
dev1
dev2

[service1]
dev1

[service2]
dev2

[service3]
dev1
dev2

And for production:

# hosts/production

prod1 ansible_ssh_host=prod1.example.com
prod2 ansible_ssh_host=prod2.example.com

[production]
prod1
prod2

[service1]
prod1

[service2]
prod2

[service3]
prod1
prod2

This allows for some nice combinations. By using ansible -i hosts I can target all known hosts. I'm using this, for example, to add all servers in the inventory to a monitoring configuration file.

By using ansible -i hosts/development I can limit the command to the development (or production) servers. I do this when I want to test a new configuration on the development system before I apply it to production.

I'm currently using this structure for about 25 servers on 3 different stages and it works pretty well for me. It has some weaknesses, though:

• Although in my case the advantages overweigh those weaknesses. My biggest fear is accidentally targeting all inventories in the hosts directory, by forgetting to limit to one of the stages. That would be easier to avoid if the inventories would be completely separate.
• Also, I don't like having to list all servers in the inventory again in the development or production groups, because it's redundant and easy to forget.
• I guess if your system grows it could become a little bit confusing to understand from where and in which order variables are loaded.

That being said, it works pretty well for me, so maybe it works well for you, too.

Extender

Is there way to avoid listing all servers in the inventory? – Burweed

@NickRoz, In some cases playbook logic requires addressing server groups directly. For example configure template on all loadbalancers servers with hostnames of all members of webservers group. In such case, maintaining multiple inventory lists manually ( repeating hostnames in groups ) would not be the best option. The following approach is recommended instead: create special group [production:children] and include other groups as members. docs.ansible.com/ansible/… – Mauritamauritania

@GregoryShulov, it's my fault, my comment was not clear enough. I use [production:children] and [development:children] groups. In case there are too many groups of servers (it happens) it is difficult to list them all again as children of production or development. Is there way to organize files somehow so as to avoid this redundancy? – Burweed

@NickRoz I don't think that's possible with static inventories. You could try to write a dynamic inventory, though: docs.ansible.com/ansible/… – Extender

@NickRoz see my answer below – Mauritamauritania

you would have 'race conditions' with this if the same variable exists in multiple files under a given environment.... – Hengelo