Remove certain fields from a YAML map object using yq

Question

I have some tricky thing that I need to do for a yaml file, this is how it looks before

Before

apiVersion: v1
items:
  - apiVersion: core.k8s.com/v1alpha1
    kind: Test
    metadata:
      creationTimestamp: '2022-02-097T19:511:11Z'
      finalizers:
        - cuv.ssf.com
      generation: 1
      name: bar
      namespace: foo
      resourceVersion: '12236'
      uid: 0117657e8
    spec:
      certificateIssuer:
        acme:
          email: myemail
      provider:
        credentials: dst
        type: foo
      domain: vst
      type: bar
    status:
      conditions:
        - lastTransitionTime: '2022-02-09T19:50:12Z'
          message: test
          observedGeneration: 1
          reason: Ready
          status: 'True'
          type: Ready
      lastOperation:
        description: test
        state: Succeeded
        type: Reconcile

https://codebeautify.org/yaml-validator/y22fe4943

I need to remove all the fields under section metadata and the tricky part is to keep only the name and namespace in addition to removing the status section at all, it should look like following

After

  apiVersion: v1
    items:
      - apiVersion: core.k8s.com/v1alpha1
        kind: Test
        metadata:
          name: bar
          namespace: foo
        spec:
          certificateIssuer:
            acme:
              email: myemail
          provider:
            credentials: dst
            type: foo
          domain: vst
          type: bar

After link https://codebeautify.org/yaml-validator/y220531ef

Using yq (https://github.com/mikefarah/yq/) version 4.19.1

Germiston

Answer 1

Using mikefarah/yq (aka Go yq), you could do something like below. Using del for deleting an entry and with_entries for selecting known fields

yq '.items[] |= (del(.status) | .metadata |= with_entries(select(.key == "name" or .key == "namespace")))' yaml

Starting v4.18.1, the eval flag is the default action, so the e flag can be avoided

Furfuran

Answer 2

This should work using yq in the implementaion of https://github.com/kislyuk/yq (not https://github.com/mikefarah/yq), and the -y (or -Y) flag:

yq -y '.items[] |= (del(.status) | .metadata |= {name, namespace})'

apiVersion: v1
items:
  - apiVersion: core.k8s.com/v1alpha1
    kind: Test
    metadata:
      name: bar
      namespace: foo
    spec:
      certificateIssuer:
        acme:
          email: myemail
      provider:
        credentials: dst
        type: foo
      domain: vst
      type: bar

Nevis

thanks, im getting an error Error: unknown shorthand flag: 'y' in -y any idea? – Germiston

we are using the latest version – Germiston

My yq has version 2.13.0. The manual says --yaml-output, --yml-output, -y : Transcode jq JSON output back into YAML and emit it. Maybe try one of the long forms. – Nevis

Alternatively, use --yaml-roundtrip, --yml-roundtrip, -Y : Transcode jq JSON output back into YAML and emit it. Preserve YAML tags and styles by representing them as extra items in their enclosing mappings and sequences while in JSON. This option is incompatible with jq filters that do not expect these extra items. – Nevis

I tried with ` yq --yaml-roundtrip '.items[] |= (del(.status) | .metadata |= {name, namespace})' ~/projects/test/a.yaml` ` and get error Error: unknown flag: --yaml-roundtrip Usage: yq eval [expression] [yaml_file1]... [flags] – Germiston

Then, I guess, something's wrong with your yq installation. Even the project's own homepage uses -y and -Y in their simple examples. – Nevis

@Nevis OP is using the Go yq – Furfuran

when I use the version I got yq --version yq (https://github.com/mikefarah/yq/) version 4.19.1 – Germiston

@Furfuran Oh, okay. Does it work without parameters, then? – Nevis

@Nevis - any idea how to make it work with the go version? – Germiston

Does it work without any parameters? – Nevis

@Nevis It tries so hard to be jq like, but not quite so yet. It used to have its own DSL, but starting to implement features like jq – Furfuran

@Alberto See my answer – Furfuran

The following should work: yq '.items[] |= (del(.status) | .metadata |= {"name": .name,"namespace":.namespace})' – Coronagraph