Ansible jinja2 escape dotted key in selectattr

Question

I am currently trying to loop over a subset of k8s_facts. My fact looks something like:

{
  "resources": [
  { 
    "metadata": {
      "annotations": {
         "com.foo.bar/name": "foo",
         "com.foo.bar/foo-name": "baz"
       },
       "creationTimestamp": "2018-12-20T02:29:50Z",
       "name": "foo-bar"
    }
  },
  ...

I want to filter on a specific value of the com.foo.bar/foo-name key. Because the key has ., - and /, it doesn't play well with the Jinja2 selectattr function. I tried to do something like that, but in vain:

- debug:
    msg: "{{ item }}"
    loop: "{{ my_fact.resources | selectattr('metadata.annotations[\\'com.foo.bar/foo-name\\']', 'defined') | selectattr('metadata.annotations[\\'com.foo.bar/foo-name\\']', 'match', 'baz') | list }}"
  loop_control: 
    label: "{{ item.metadata.name }}"

When executing the previous, I get this error:

fatal: [<redacted>]: FAILED! => {"msg": "template error while templating string: expected token ',', got 'com'. String: {{ my_fact.resources | selectattr('metadata.annotations[\\\\'com.foo.bar/foo-name\\\\']', 'defined') | selectattr('metadata.annotations[\\\\'com.foo.bar/foo-name\\\\']', 'match', 'baz') | list }}"}

My question is, how can I escape complex strings containings dots in Jinja2?

Horton

Answer 1

When I'm entering escaping hell in ansible, I tend to take advantage of the yaml folded and literal block syntax. The advantage is that it lets you write jinja blocks without having to surround them with quotes, which eliminates one quoting level hence one escaping level as well.

In your case I think you can go straight to the point by using the json_query filter rather than piping a long list of filters.

Here is a demo playbook:

---
- name: Test var names with dots
  hosts: localhost
  gather_facts: false

  vars:
    my_fact: {
      "resources": [
        {
          "metadata": {
            "annotations": {
              "com.foo.bar/name": "foo",
              "com.foo.bar/foo-name": "baz"
            },
            "creationTimestamp": "2018-12-20T02:29:50Z",
            "name": "foo-bar"
          }
        },
        {
          "metadata": {
            "annotations": {
              "com.foo.bar/name": "toto",
              "com.foo.bar/foo-name": "titi"
            },
            "creationTimestamp": "2018-12-21T02:30:50Z",
            "name": "foo-bla"
          }
        },
        {
          "metadata": {
            "annotations": {
              "com.foo.bar/name": "johnsmith",
              "com.foo.bar/foo-name": "baz"
            },
            "creationTimestamp": "2018-12-22T02:31:50Z",
            "name": "foo-john"
          }
        }
      ]
    }

  tasks:
    - name: Show results where metadata.annotations."com.foo.bar/foo-name"=='baz'
      vars:
        query: >-
          [?(metadata.annotations."com.foo.bar/foo-name")=='baz']
      debug:
        msg: "{{ item }}"
      loop: "{{ my_fact.resources | json_query(query) }}"
      loop_control:
        label: "{{ item.metadata.name }}"

And the result

PLAY [Test var names with dots] *******************************************************************************************************************************************************************************************

TASK [Show results where metadata.annotations."com.foo.bar/foo-name"=='baz'] **********************************************************************************************************************************************
ok: [localhost] => (item=foo-bar) => {
    "msg": {
        "metadata": {
            "annotations": {
                "com.foo.bar/foo-name": "baz",
                "com.foo.bar/name": "foo"
            },
            "creationTimestamp": "2018-12-20T02:29:50Z",
            "name": "foo-bar"
        }
    }
}
ok: [localhost] => (item=foo-john) => {
    "msg": {
        "metadata": {
            "annotations": {
                "com.foo.bar/foo-name": "baz",
                "com.foo.bar/name": "johnsmith"
            },
            "creationTimestamp": "2018-12-22T02:31:50Z",
            "name": "foo-john"
        }
    }
}

PLAY RECAP ****************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Gamelan

Answer 2

Q: "I want to filter on a specific value of the com.foo.bar/foo-name key"

A: The below json_query does the job

  q1: '[?metadata.annotations."com.foo.bar/foo-name"==`baz`]'
  r1: "{{ resources|json_query(q1) }}"

gives

  r1:
  - metadata:
      annotations:
        com.foo.bar/foo-name: baz
        com.foo.bar/name: foo
      creationTimestamp: '2018-12-20T02:29:50Z'
      name: foo-bar

---

Example of a complete playbook for testing

- hosts: all

  vars:
    resources:
      - metadata:
          annotations:
            com.foo.bar/foo-name: baz
            com.foo.bar/name: foo
          creationTimestamp: '2018-12-20T02:29:50Z'
          name: foo-bar

    q1: '[?metadata.annotations."com.foo.bar/foo-name"==`baz`]'
    r1: "{{ resources|json_query(q1) }}"

  tasks:

    - debug:
        var: r1

Serrate

I guess this would work if I would only want to display the tag value. I want to filter the facts but keep the objects whole. – Horton

In fact, this will work and keep the objects intact. – Earring

Answer 3

The short answer: with only | selectattr, you can't.

Here is the relevant code in Jinja:

def _prepare_attribute_parts(
    attr: t.Optional[t.Union[str, int]]
) -> t.List[t.Union[str, int]]:
    if attr is None:
        return []

    if isinstance(attr, str):
        return [int(x) if x.isdigit() else x for x in attr.split(".")]

    return [attr]

There doesn't appear to be a way to do any escaping, or indeed even a way to pass a Python tuple or array in lieu of the string to be split on ..

My recommendation is to write your own filter plugin instead, which is easy enough.

Postconsonantal