Reload Ansible's dynamic inventory

Question

I'm using Ansible to setup EC2 instances and deploy an application. There's a hosts script which gathers tags related servers and groups info. I'd like to run these actions as a single playbook, so

1. New instances are created if needed
2. Hosts script loads inventory (including servers' facts)
3. Deployment playbook works

However, the inventory is loaded in advance, so, there is no servers/groups data if servers are created/updated during the play. I can

1. separate provision and deployment playbooks
2. use add_host trick to emulate dynamic inventory when servers are updated

But, there are drawbacks in those approaches.

Can I force Ansible to reload inventory?
My test files are: hosts script:

#!/bin/sh
echo `date` >> log.log
echo "{\"standalone\":[\"localhost\"]}"

Sample playbook.yml:

---
- hosts: all
  tasks:
    - name: show inventory_hostname
      command: echo {{ inventory_hostname }}

I run it with the command ansible-playbook -i hosts playbook.yml -v and see two runs:

$> cat log.log
Thu Mar 12 09:43:16 SAMT 2015
Thu Mar 12 09:43:16 SAMT 2015

but I haven't found a command to double it.

Afflictive

What drawback do you see with add_host module? – Correy

@Correy It's a logic duplication, isn't it? I don't want to think about cases when all servers brand new, or when they are changed partly (my distribution model includes several servers). Moreover, hosts script generate some local facts, should I copy it as well? – Afflictive

I agree with Vladimir @Mxx. Using add_host here is duplicative. I am provisioning a number of hosts based on a list, but this breaks the "register: ec2" pattern that's common in ansible ec2 provisioning examples, because the result is a list of return values. Ideally ansible would allow me to run a two-phased provisioning process in the same playbook: 1) create a bunch of instances, then 2) address them by group and do some basic configuration (like setting up DNS, etc). – Pains

Can confirm: add_host is suboptimal to say the least. – Biggin

Answer 1

With Ansible 2.0+, you can refresh your inventory mid-play by running the task:

- meta: refresh_inventory

Parthinia

This was really helpful. I ended up adding it as a post_task to my playbook and then continued with other plays. Super useful. There isn't much documentation on this, but if you check out the github repo for ansible you will see it there. Another way to achieve this would be running your ec2.py file with --refresh-cache. – Timothea

Does refresh_inventory really work though? I'm using it in my playbooks and there's no pause to indicate that it's refreshing the inventory, and more to the point, it's not discovering the ec2 instance that has been created since the initial inventory collection. I've tested giving it more time manually with pausing incase it was a matter of the instance still initialising but... that doesn't seem to help. – Devisal

For anyone else, I think I found my issue. Meta: refresh inventory is not like using ./ec2.py --refresh-cache which forces a cache refresh. I found that I had to lower (or more specifically, disable entirely) the inventory caching. You do this by setting cache_max_age = 0 in ec2.ini. – Devisal

This i.e., meta: refresh_inventory does not work for vCenter inventory. Any idea on how to refresh VMware dynamic inventory? – Thegn

This seems to work well with the aws_ec2 inventory source, without needing the extra --refresh-cache that the old ec2.py dynamic inventory method apparently did require. – Squamosal

This does not seem to work for Google Cloud dynamic inventory either specifically the gcp_compute.py inventory plugin. I had to resort to add_host & mimic gcp labeling with groups parameter. suboptimal :( – Edytheee

Answer 2

I found the meta: refresh_inventory to be insufficient.
I had to add an explicit call to ec2.py --refresh-cache first.

- name: refresh inventory
  hosts: localhost
  connection: local
  gather_facts: False
  tasks:
    - name: Refresh EC2 cache
      command: /etc/ansible/ec2.py --refresh-cache
    - name: Refresh in-memory EC2 cache
      meta: refresh_inventory

Redhot

This worked for me with rax.py using inventory/rax.py --list --refresh-cache – Pharyngeal

to make the ec2.py location more flexible (for example when using multiple inventories), you can use {{inventory_dir}}/ec2.py --refresh-cache – Melnick

Thanks for posting this, we are still seeing inconsistent results with this solution, the first execution does get the right inventory and is output in Ansible but the in-memory refresh doesn't seem to have the right hosts, only some times. My colleague @eric-oliver just opened an issue at github.com/ansible/ansible/issues/71169 as we found some source code in the refresh_inventory code that says ` # FIXME: flush inventory cache `. So follow that issue for more on that. – Pimp

Answer 3

Ansible currently doesn't support this. If you look at the source code of the ansible or ansible-playbook commands you'll see that the inventory is loaded first and then the inventory object is passed to the ansible command that runs the specified task or playbook. Moving the inventory processing so that it happens within the task/playbook handlers would probably be a pretty major undertaking for a number of reasons.

Your best bet when doing something like this is to simply break your playbook into two and wrap their calls in a shell script that you only have to invoke once.

Handicraftsman

This comment should be modified to tell it predates version 2.0 of ansible when reloading inventory was not possible, or deleted. – Lehr

Answer 4

You can also edit the ec2.ini file and set the option:

cache_max_age = 0

to prevent the need for reload by making sure that nothing is cached in the first place.

Michaelmichaela

Answer 5

I had a similar scenario and meta did not help, but setup did. So, this is for all folks that might find this useful... To give a little bit of context, what I was doing is to limit the linux kernel memory to a certain amount of what the real memory was and then update some files based on the new values.

For instance, the ansible tasks were something like:

- name: Print real memory
  debug:
    msg: {{ ansible_memory_mb.real.total }}

- name: Reset GRUB_CMDLINE_LINUX
  lineinfile:
    path: /etc/default/grub
    regexp: '^{{ item.key }}='
    line: '{{ item.key }}="{{ item.value }}"'
  with_items:
    - { key: 'GRUB_CMDLINE_LINUX', 
        value: '{{ ansible_memory_mb.real.total - reserved }}m'}

- name: Update GRUB
  command: update-grub

- name: Reboot the node to load the system's real memory settings
  reboot:

- name: Print real memory
  debug:
    msg: {{ ansible_memory_mb.real.total }}

and the output was the same for both debug logs because the ansible_memory_mb variable was not reload after the linux kernel configuration changed.

Solution? Add a task to setup after rebooting the system.

- name: Refresh ansible facts (ansible_memory_mb) after updating grub
  setup:

Ned

Answer 6

Please note that meta: refresh_inventory does not work in Ansible Tower (AWX).

https://docs.ansible.com/ansible-tower/3.3.1/html/userguide/best_practices.html

Playbooks should not use the meta: refresh_inventory Ansible feature, as it is incompatible with how Tower processes inventory, if a playbook does refresh inventory, it will refresh back to the start of the inventory the job began with. Instead, use a workflow with separate inventory refresh steps.

I do not yet have enough rep to comment, otherwise, I would have.

Kirkman

Answer 7

Take a look at add_host.

It does add a host (and alternatively a group) to the ansible-playbook in-memory inventory

Uziel

As I said - add_host is an option which I dislike (see comments above for details) – Afflictive