JQ How to merge multiple objects into one
Asked Answered
D

1

8

Given the following input (which is a toned down version of the output with 100K+ objects of another complex query):

echo '{ "a": { "b":"c", "d":"e" } }{ "a": { "b":"f", "d":"g" } }' | jq '.'
{
  "a": {
    "b": "c",
    "d": "e"
  }
}
{
  "a": {
    "b": "f",
    "d": "g"
  }
}

desired output:

{
   "c": "e",
   "f": "g"
}

or (suits better for follow up usage):

{
   x: {
      "c": "e",
      "f": "g"
   }
}

I can't for the life of me figure out how to do it. My real problem of course is the multiple object input data, for which I really don't know whether it's valid JSON. Jq produces and accepts it, jshon does not. I tried various possibilities, but none of them produced what I need. I considered this the most likely candidate:

echo '{ "a": { "b":"c", "d":"e" } }{ "a": { "b":"f", "d":"g" } }' | jq ' . | { (.a.b): .a.d }'
{
   "c": "e"
}
{
   "f": "g"
}

But alas. Other things I tried:

' . | { x: { (.a.b): .a.d } }'
'{ x: {} | . | add }'
'{ x: {} | . | x += }'
'{ x: {} | x += . }'
'x: {} | .x += { (.a.b): .a.d }'
'{ x: {} } | .x += { (.a.b): .a.d }'

Another one, close, but no sigar:

'reduce { (.a.b): .a.d } as $item ({}; . + $item)'
{
  "c": "e"
}
{
  "f": "g"
}

Who cares to enlighten me?

So the full answer in the above use case, thanks to @peak, is

echo '{ "a": { "b": "c", "d": "e" } }{ "a": { "b": "f", "d": "g" } }' | jq -n '{ x: [inputs | .a | { (.b): .d} ] | add }'
{
  "x": {
    "c": "e",
    "f": "g"
  }
}
Drus answered 24/7, 2018 at 5:49 Comment(0)
C
9

Let's assume you have jq 1.5 or later, and that your JSON objects are in one or more files. Then for your first expected output you can simply write:

[inputs | .a | { (.b): .d} ] | add

This assumes you use the -n command-line option. I'm sure that once you see how simple the solution is, further explanation will be superfluous.

For your second expected output, you can just wrap the above line in:

{x: _}

E.g.:

$ jq -ncf program.jq input.json
{"x":{"c":"e","f":"g"}}

p.s. Your approach using reduce is fine, but you'd need to use the -s command-line option.

p.p.s. Do you owe me another beer?

Counterpoise answered 24/7, 2018 at 7:11 Comment(5)
You're close, closer than I have been until now, I have to admit that, but the input is a single file :-( (PS. Are going for a full crate? ;-) )Drus
As I said, the solution with inputs will work no matter how many files there are.Counterpoise
Nope, the ';' gives a syntax error (am on linux). Leaving it out gives { "f": "g" }, not { "c": "e", "f": "g" } :-(Drus
The ; was a typo. Fixed.Counterpoise
Leaving out the input, the result still is | jq '[inputs | .a | { (.b): .d} ] | add' { "f": "g" } not { "c": "e", "f": "g" }Drus

© 2022 - 2024 — McMap. All rights reserved.