How can I output the whole document in jq while replacing an item based on a field's value?
Asked Answered
B

2

5

I am attempting to use jq to parse a AWS CloudFront configuration JSON file and change a few values so that I can then issue an update statement with that configuration.

The Document

The document format snippet that matters, with redacted values, is:

{
    "ETag": "REDACTED",
    "DistributionConfig": {
        "Origins": {
            "Quantity": 2,
            "Items": [
                {
                    "Id": "redacted-1",
                    "DomainName": "redacted1.us-east-1.amazonaws.com",
                    "OriginPath": "/redacted"
                },
                {
                    "Id": "redacted-2",
                    "DomainName": "redacted2.s3.amazonaws.com",
                    "OriginPath": ""
                }
            ]
        }
    }    
}

What I'm trying to do

I want to output the entire document, but:

  • I want to set the ETag value to ""
  • On the 2nd item in the items array, I want to set the OriginPath to a value of my choosing

(this is to support our CI/CD process being able to point the CloudFront distribution to a new folder of just-deployed code within an S3 bucket. I want to modify the existing configuration in those specific ways but leave the rest in-tact.)

Something that works...sort of

A jq filter of . | (.DistributionConfig.Origins.Items[1].OriginPath = "Hello") | .ETag = "" does what I need it to do, resulting in:

{
  "ETag": "", // correctly updated
  "DistributionConfig": {
    "Origins": {
      "Quantity": 2, // correctly retained
      "Items": [
        {
          "Id": "redacted-1",
          "DomainName": "redacted1.us-east-1.amazonaws.com",
          "OriginPath": "/redacted"
        },
        {
          "Id": "redacted-2",
          "DomainName": "redacted2.s3.amazonaws.com",
          "OriginPath": "Hello" // correctly updated
        }
      ]
    }
  }
}

Where I'm having trouble

The solution above works...as long as I'm referring to the 2nd item in the array. But I'm not always sure it will be the second item in the array.

So instead, I'd like to match based on the Id property.

Any solutions I've been able to find that do this seem to filter the document to that section of JSON, rather than matching and updating the value as part of outputting the document.

Question

How can I filter an array for a field that has a certain value, and still output the whole document?

Or put another way -- given the document above, how can I change:

. | (.DistributionConfig.Origins.Items[1].OriginPath = "Hello") | .ETag = ""

To something that lets me refer to .Id="redaacted-2" rather than Items[1]?

A link to a demo

Demo of the script at https://jqplay.org/s/ZQ7XcM5-BY in case anyone wants to try an answer.

Busiek answered 21/6, 2021 at 20:2 Comment(0)
A
5

Just a minor variation on @peak's answer, using select to choose which item(s) to update:

.ETag = ""
| (.DistributionConfig.Origins.Items[] | 
      select(.Id == "redacted-2")
  ).OriginPath = "foo"

The second assignment receives the entire original input as its input, so produces it as its output. The parenthesized filter, though, selects only the appropriate element(s) of Items whose OriginPath key you want to update.

Acetylene answered 21/6, 2021 at 20:49 Comment(1)
This ended up being the exact solution I was looking for; the issue I was having is that I was not appropriately understanding the nature of the parenthesized filtering.Busiek
H
7

The basic way to accomplish edits such as these is using the syntax:

PATHSPEC = VALUE

or if the value depends on PATHSPEC in a certain way:

PATHSPEC |= VALUE

where PATHSPEC is a jq path specification.

Using this principle, in your case, you could write:

.ETag = ""
| .DistributionConfig.Origins.Items[1].OriginPath = "myvalue"

or, if you want to base the second update on .Id:

.ETag = ""
| .DistributionConfig.Origins.Items[] |=
   if .Id == "redacted-2" then .OriginPath = "myvalue" else . end

The example immediately above can be seen in action at https://jqplay.org/s/u5xbhbSs4l

Variations

There are of course numerous variations. For example, you could use .Quantity as the index of the item to be updated:

.ETag = ""
| .DistributionConfig.Origins.Quantity as $ix
| .DistributionConfig.Origins.Items[$ix - 1].OriginPath = "myvalue"

or more DRYly:

.ETag = ""
| .DistributionConfig.Origins |=
    (.Items[.Quantity  - 1].OriginPath = "myvalue")

Harlandharle answered 21/6, 2021 at 20:36 Comment(4)
I'm not sure I was clear. My concern is that rather than Items[1], I need to effectively say "Items where the Id field is redacted-2. Using the array notation currently works, but I cannot guarantee the item I need will be in that position, so I need to go based on the ID field.Busiek
Thank you for digging in on this! I am unable to produce a result in my example using your answers. I added a link to a jqplay demo. My goal is to achieve exactly what that example does, but using the Id field instead of Items[1].Busiek
Please see the link to jqplay that I added just now.Harlandharle
Thank you! @chepner's answer ended up being exactly spot on for what I was looking for, but I've given you an up-vote and appreciate the depth of your answer!Busiek
A
5

Just a minor variation on @peak's answer, using select to choose which item(s) to update:

.ETag = ""
| (.DistributionConfig.Origins.Items[] | 
      select(.Id == "redacted-2")
  ).OriginPath = "foo"

The second assignment receives the entire original input as its input, so produces it as its output. The parenthesized filter, though, selects only the appropriate element(s) of Items whose OriginPath key you want to update.

Acetylene answered 21/6, 2021 at 20:49 Comment(1)
This ended up being the exact solution I was looking for; the issue I was having is that I was not appropriately understanding the nature of the parenthesized filtering.Busiek

© 2022 - 2025 — McMap. All rights reserved.