Merging Two JSON Documents Using Jackson

Question

Is it possible to merge two JSON documents with the Jackson JSON library? I am basically using the Jackson mapper with simple Java Maps.

I've tried to search in Google and Jackson's documentation but couldn't find anything.

Leveridge

Answer 1

One way is to use ObjectReader like so:

MyBean defaults = objectMapper.readValue(defaultJson, MyBean.class);
ObjectReader updater = objectMapper.readerForUpdating(defaults);
MyBean merged = updater.readValue(overridesJson);

which will combine data from two sources. This only makes a shallow copy, i.e. does not do recursive merge on contained objects.

Otherwise you may need to just read JSON as a tree (JsonNode), loop over contents and merge manually. This often makes sense anyway since rules of merging are not trivial, and everyone has their own ideas of how merging should work.

EDIT: (03-Apr-2017)

As per @Fernando Correia's comment, there is actually a new feature added in upcoming Jackson 2.9 (to be released in April or May 2017) that does allow deep merging, finally.

Woehick

Thank you for your answer. I do need a deep merge, hence you're suggestion to write the merge manually echos the gut feeling I had when I posted the question. My underlying data structures are Maps, hence it should be worth writing a generic deep merge routine for java Maps. It may prove to be re-usable in some future project. – Leveridge

is deep merge available as a feature now ? – Mallory

No, not as of Jackson 2.6. – Woehick

It is implemented for the (upcoming) version 2.9: github.com/FasterXML/jackson-databind/issues/1399 – Aardwolf

@FernandoCorreia well spotted, it is indeed! Thank you for adding an update; I will add an edit to my answer. – Woehick

Answer 2

Inspired by StaxMans answer I implemented this merging method.

public static JsonNode merge(JsonNode mainNode, JsonNode updateNode) {

    Iterator<String> fieldNames = updateNode.fieldNames();
    while (fieldNames.hasNext()) {

        String fieldName = fieldNames.next();
        JsonNode jsonNode = mainNode.get(fieldName);
        // if field exists and is an embedded object
        if (jsonNode != null && jsonNode.isObject()) {
            merge(jsonNode, updateNode.get(fieldName));
        }
        else {
            if (mainNode instanceof ObjectNode) {
                // Overwrite field
                JsonNode value = updateNode.get(fieldName);
                ((ObjectNode) mainNode).put(fieldName, value);
            }
        }

    }

    return mainNode;
}

Hope this helps someone.

Fleet

Looks pretty good @Arn! It will be helpful if I need to do this in a future project. When I asked this question, I was working with elasticsearch and I found a partial update plugin for it. That helped me at that time. – Leveridge

Excellent, Arne, that is exactly what I've needed, you've saved me hours of investigation, thank you. To make this a little more than thank you comment, as of Jackson 2.4 put() is deprecated and should be substituted with replace(). Also, for JDK8 users, the code could be made more concise with trivial invoking of forEachRemaining() directly on the iterator and passing in the somewhat shorter lambda-expression. – Exerciser

great thanks. does this work for embedded arrays? both when adding elements to the array and when removing elements ? or do we have to replace the whole array? – Cohosh

Answer 3

Inspired by Arn's answer. Editing it to add the case where a node may have a array of nodes in it.

public static JsonNode merge(JsonNode mainNode, JsonNode updateNode) {

    Iterator<String> fieldNames = updateNode.fieldNames();

    while (fieldNames.hasNext()) {
        String updatedFieldName = fieldNames.next();
        JsonNode valueToBeUpdated = mainNode.get(updatedFieldName);
        JsonNode updatedValue = updateNode.get(updatedFieldName);

        // If the node is an @ArrayNode
        if (valueToBeUpdated != null && valueToBeUpdated.isArray() && 
            updatedValue.isArray()) {
            // running a loop for all elements of the updated ArrayNode
            for (int i = 0; i < updatedValue.size(); i++) {
                JsonNode updatedChildNode = updatedValue.get(i);
                // Create a new Node in the node that should be updated, if there was no corresponding node in it
                // Use-case - where the updateNode will have a new element in its Array
                if (valueToBeUpdated.size() <= i) {
                    ((ArrayNode) valueToBeUpdated).add(updatedChildNode);
                }
                // getting reference for the node to be updated
                JsonNode childNodeToBeUpdated = valueToBeUpdated.get(i);
                merge(childNodeToBeUpdated, updatedChildNode);
            }
        // if the Node is an @ObjectNode
        } else if (valueToBeUpdated != null && valueToBeUpdated.isObject()) {
            merge(valueToBeUpdated, updatedValue);
        } else {
            if (mainNode instanceof ObjectNode) {
                ((ObjectNode) mainNode).replace(updatedFieldName, updatedValue);
            }
        }
    }
    return mainNode;
}

Sedgewinn

The casting of valueToBeUpdated to an ArrayNode is not safe. You can get a TextNode, for instance, instead of an ArrayNode in that variable. – Sruti

Thanks PNS. I have edited my answer and added the condition to check if valueToBeUpdated is an arrayNode – Sedgewinn

Cool - but doesn't work if updateNode starts with an ArrayNode, because they do not have fields, just children. BTW - ObjectReader updating work for me. – Lavolta

Thanks to you I'm going to sleep more hour today, thank you my friend!! :) – Sophy

Thanks a ton for this. Works like a charm. Btw it doesn't work if root node is JSONArray, so I just modified the code by copying while-loop's ArrayNode code at top level in if-else – Fiducial

Answer 4

Below is an implementation in Scala. The source and target node are mostly commutative except when a branch exists in both source and target.

  def mergeYamlObjects(source: ObjectNode, target: ObjectNode, overwrite: Boolean = true): ObjectNode = {
    if (target == null)
      source
    else if (source == null)
      target
    else {
      val result = source.deepCopy
      val fieldlist = source.fieldNames.asScala.toList ++ target.fieldNames.asScala.toList
      for (item <- fieldlist) {
        if (!(source has item)) {
          result put(item, target get item)
        } else {
          if ((source get item).isValueNode) {
            if (target has item)
              if (overwrite)
                result.put(item, target get item)
          } else {
            result.put(item, mergeYamlObjects(source.get(item).asInstanceOf[ObjectNode],
              target.get(item).asInstanceOf[ObjectNode], overwrite = overwrite))
          }
        }
      }
      result
    }
  }

Delphinium

Answer 5

If someone simply wants to add two or more JsonNode object into one JsonNode, this can be one approach:

ArrayNode arrayNode = objectMapper.createArrayNode();
arrayNode.add(firstJsonNode);
arrayNode.add(secondJsonNode);
arrayNode.add(thirdJsonNode);

JsonNode root = JsonNodeFactory.instance.objectNode();
((ObjectNode) root).put("", arrayNode);
System.out.println("merged array node #: " + root);

Lobo

The method put(String, JsonNode) from the type ObjectNode is deprecated , use set or replace – Lighting

Answer 6

Here,is full implementation of merging two JSON tree into one. Hope it would be helpful :)

/**
 * Merge two JSON tree into one i.e mergedInTo.
 *
 * @param toBeMerged
 * @param mergedInTo
 */
public static void merge(JsonNode toBeMerged, JsonNode mergedInTo) {
    Iterator<Map.Entry<String, JsonNode>> incomingFieldsIterator = toBeMerged.fields();
    Iterator<Map.Entry<String, JsonNode>> mergedIterator = mergedInTo.fields();

    while (incomingFieldsIterator.hasNext()) {
        Map.Entry<String, JsonNode> incomingEntry = incomingFieldsIterator.next();

        JsonNode subNode = incomingEntry.getValue();

        if (subNode.getNodeType().equals(JsonNodeType.OBJECT)) {
            boolean isNewBlock = true;
            mergedIterator = mergedInTo.fields();
            while (mergedIterator.hasNext()) {
                Map.Entry<String, JsonNode> entry = mergedIterator.next();
                if (entry.getKey().equals(incomingEntry.getKey())) {
                    merge(incomingEntry.getValue(), entry.getValue());
                    isNewBlock = false;
                }
            }
            if (isNewBlock) {
                ((ObjectNode) mergedInTo).replace(incomingEntry.getKey(), incomingEntry.getValue());
            }
        } else if (subNode.getNodeType().equals(JsonNodeType.ARRAY)) {
            boolean newEntry = true;
            mergedIterator = mergedInTo.fields();
            while (mergedIterator.hasNext()) {
                Map.Entry<String, JsonNode> entry = mergedIterator.next();
                if (entry.getKey().equals(incomingEntry.getKey())) {
                    updateArray(incomingEntry.getValue(), entry);
                    newEntry = false;
                }
            }
            if (newEntry) {
                ((ObjectNode) mergedInTo).replace(incomingEntry.getKey(), incomingEntry.getValue());
            }
        }
        ValueNode valueNode = null;
        JsonNode incomingValueNode = incomingEntry.getValue();
        switch (subNode.getNodeType()) {
            case STRING:
                valueNode = new TextNode(incomingValueNode.textValue());
                break;
            case NUMBER:
                valueNode = new IntNode(incomingValueNode.intValue());
                break;
            case BOOLEAN:
                valueNode = BooleanNode.valueOf(incomingValueNode.booleanValue());
        }
        if (valueNode != null) {
            updateObject(mergedInTo, valueNode, incomingEntry);
        }
    }
}

private static void updateArray(JsonNode valueToBePlaced, Map.Entry<String, JsonNode> toBeMerged) {
    toBeMerged.setValue(valueToBePlaced);
}

private static void updateObject(JsonNode mergeInTo, ValueNode valueToBePlaced,
                                 Map.Entry<String, JsonNode> toBeMerged) {
    boolean newEntry = true;
    Iterator<Map.Entry<String, JsonNode>> mergedIterator = mergeInTo.fields();
    while (mergedIterator.hasNext()) {
        Map.Entry<String, JsonNode> entry = mergedIterator.next();
        if (entry.getKey().equals(toBeMerged.getKey())) {
            newEntry = false;
            entry.setValue(valueToBePlaced);
        }
    }
    if (newEntry) {
        ((ObjectNode) mergeInTo).replace(toBeMerged.getKey(), toBeMerged.getValue());
    }
}

Arber

Hi , maybe is better to use a deepCopy on each getValue ? – Resilience

Answer 7

If your goal is to concatenate two JSONs, the simplest way I found is the following (considering you have two ObjecNode ready):

ObjectMapper mapper = new ObjectMapper();

// Dummy objects to concatenate
Map<String, Object> map = new HashMap();
map.put("k", "v");

Map<String, Object> secondMap = new HashMap();
secondMap.put("secondK", "secondV");

//Transforming Objects into ObjectNode
ObjectNode firstObjectNode = mapper.convertValue(map, ObjectNode.class);
ObjectNode secondObjectNode = mapper.convertValue(secondMap, ObjectNode.class);

//Concatenating secondObjectNode into firstObjectNode
firstObjectNode.setAll(secondObjectNode);

//Output will be: {"k":"v","secondK":"secondV"}
System.out.println(firstObjectNode);

Hope it helps

Yesterday

Answer 8

Merge json -> json with ObjectMapper:

@SneakyThrows
public static String mergeToPretty(String baseJson, String overrideJson) {
    ObjectMapper objectMapper = new ObjectMapper();
    
    Object base = objectMapper.readValue(baseJson, Object.class);
    Object merged = objectMapper.readerForUpdating(base).readValue(overrideJson);
    return objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(merged);
}

Notes:

• @SneakyThrows could be replaced with exception handling if not using Lombok.
• If no need for pretty, remove .writerWithDefaultPrettyPrinter() from last line.

Brynhild