JAXB/Moxy Unmarshalling assigns all field values to Map<String,Object> rather than the specific field provided for it

Question

In short, I would like to perform the unmarshalling as mentioned here but along with Map I will have one more @XmlElement. So one field is annotated with (Map field) @XmlPath(".") and another field with (String field) @XmlElement and then I would like to perform unmarshalling.

My main goal of the application is to convert XML->JSON and JSON->XML using the JAXB/Moxy and Jackson library. I am trying to unmarshal the XML and map it to the Java POJO. My XML can have some dedicated elements and some user-defined elements which can appear random so I would like to store them in Map<String, Object>. Hence, I am making use of XMLAdapter. I am following the blog article to do so. I am not doing exactly the same but a bit different.

The problem I am facing is during unmarshalling the dedicated fields are not taken into consideration at all. All the values are unmarshalled to Map<String.Object>. As per my understanding it's happening because of the annotation @XmlPath(".") and usage of XMLAdapter but If I remove this annotation then it won't work as expected. Can someone please help me with this issue? The marshaling works fine with both @XmlPath(".") and XMLAdapter. The problem is arising only during unmarshalling.

Following is my XML that I would like to convert to JSON: (Note: Name and Age are dedicated fields and others is the user-defined field.)

<Customer xmlns:google="https://google.com">
  <name>BATMAN</name>
  <age>2008</age>
  <google:main>
    <google:sub>bye</google:sub>
  </google:main>
</Customer>

Following is my Customer class used for marshaling, unmarshalling by Moxy and Jackson: (Note: Name and Age are dedicated fields and others is the user-defined field. I want others to store only the values that cannot be mapped directly to POJO such as google:main and its children from above XML)

@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "others"})
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer {
  private String name;
  private String age;

  @XmlPath(".")
  @XmlJavaTypeAdapter(TestAdapter.class)
  private Map<String, Object> others;
  //Getter, Setter and other constructors
}

Following is my TestAdapter class which will be used for the Userdefined fields:

class TestAdapter extends XmlAdapter<Wrapper, Map<String, Object>> {

  @Override
  public Map<String, Object> unmarshal(Wrapper value) throws Exception {
    System.out.println("INSIDE UNMARSHALLING METHOD TEST");
    final Map<String, Object> others = new HashMap<>();

    for (Object obj : value.getElements()) {
      final Element element = (Element) obj;
      final NodeList children = element.getChildNodes();

      //Check if its direct String value field or complex
      if (children.getLength() == 1) {
        others.put(element.getNodeName(), element.getTextContent());
      } else {
        List<Object> child = new ArrayList<>();
        for (int i = 0; i < children.getLength(); i++) {
          final Node n = children.item(i);
          if (n.getNodeType() == Node.ELEMENT_NODE) {
            Wrapper wrapper = new Wrapper();
            List childElements = new ArrayList();
            childElements.add(n);
            wrapper.elements = childElements;
            child.add(unmarshal(wrapper));
          }
        }
        others.put(element.getNodeName(), child);
      }
    }

    return others;
  }

  @Override
  public Wrapper marshal(Map<String, Object> v) throws Exception {
    Wrapper wrapper = new Wrapper();
    List elements = new ArrayList();
    for (Map.Entry<String, Object> property : v.entrySet()) {
      if (property.getValue() instanceof Map) {
        elements.add(new JAXBElement<Wrapper>(new QName(property.getKey()), Wrapper.class, marshal((Map) property.getValue())));
      } else {
        elements.add(new JAXBElement<String>(new QName(property.getKey()), String.class, property.getValue().toString()));
      }
    }
    wrapper.elements = elements;
    return wrapper;
  }
}

@Getter
class Wrapper {

  @XmlAnyElement
  List elements;
}

And finally, my Main class will be used for marshaling and unmarshalling. Also, to convert to JSON and XML.

class Main {

  public static void main(String[] args) throws JAXBException, XMLStreamException, JsonProcessingException {

    //XML to JSON
    JAXBContext jaxbContext = JAXBContext.newInstance(Customer.class);
    Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
    InputStream inputStream = Main.class.getClassLoader().getResourceAsStream("Customer.xml");
    final XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
    final XMLStreamReader streamReader = xmlInputFactory.createXMLStreamReader(inputStream);
    final Customer customer = unmarshaller.unmarshal(streamReader, Customer.class).getValue();
    final ObjectMapper objectMapper = new ObjectMapper();
    final String jsonEvent = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(customer);
    System.out.println(jsonEvent);

    //JSON to XML
    Marshaller marshaller = jaxbContext.createMarshaller();
    marshaller.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
    marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
    marshaller.marshal(customer, System.out);
  }
}

When I convert the XML->JSON then I get the following output: (If you observe the fields name and age are not taken as the dedicated fields from Customer class rather its taken as random fields and written within the others)

{
  "name" : "",
  "age" : "",
  "others" : {
    "google:main" : [ {
      "google:sub" : "bye"
    } ],
    "name" : "BATMAN",
    "age" : "2008"
  }
}

I want my output to be something like this: (I want my dedicated fields to be mapped first then if there are any unknown fields then map them later within others MAP). Please note that I do not want to get others tag within my JSON. I want to get the names of the fields only for the dedicated fields.

{
  "name": "BATMAN",
  "age": 2008,
  "google:main": {
    "google:sub": "bye"
  }
}

Following is the XML that I would like to get during the marshaling. Also, please note I am using @XmlPath(".") so that I do not get the others node within my XML during marshaling.

<Customer>
    <name>BATMAN</name>
    <age>2008</age>
    <google:main>>
        <google:sub>bye</google:sub>
    </google:main>
</Customer>

The marshaling is working fine. The problem is happening during .unmarshaling As per my understanding it's happening because of the annotation @XmlPath(".") with XMLAdapter but If I remove this annotation then it won't work as expected. Can someone please help me with this issue?

** Edited **

I thought of a few workarounds but nothing seems to work for me. They are getting messed up due to @XmlPath("."). Still looking for some idea or workarounds. Any help would be really appreciated.

Liman

Even after trying a lot of things and debugging unable to find the resolution or workaround. If anyone has some solution please suggest something. – Liman

The answer provided by @Anhydrite did not work for me. Still looking for some suggestion or workaround. Please help. – Liman

Instead of XmlPath("."), try Customer/*[not(self::name | self::age)] or *[not(self::name | self::age)] – Nighttime

@LuisMuñoz Thanks a lot for taking your time and responding. Do you mean replace @XmlPath(".") with @XmlPath(""Customer/*[not(self::name | self::age)])? or replace the whole @XmlPath(".") with Customer/*[not(self::name | self::age)]? I am a bit confused can you please provide an example or guide me to some documentation where these things are covered. Thanks in advance. – Liman

@LuisMuñoz I tried few things based on your answer but it did not work. I tried looking for some documentation but could not find anything. Maybe I am not following the right approach. If possible please provide an example or documentation link that would be really helpful for me. – Liman

Tried myself and didn't work, yet. – Nighttime

@LuisMuñoz Thanks for the response and efforts. Any other workaround or suggestions? I am really confused. Tried to debug the code but the code seems to be very complex for my JAVA knowledge so unable to follow what's happening in there so unable pin point what exactly could be wrong. If possible please suggest something. Thanks in advance. – Liman

Posted a workaround as an answer. Hope it helps :-) – Nighttime

Answer 1

ah, finally some relief. This issue ate my head a lot but I was finally able to find a workaround. Tried a lot of things and reached out to many people but nothing seems to work and I thought it's an issue from the JAXB/Moxy library. I was able to find a workaround. Hope it helps someone in the future and do not get frustrated like me :)

I used 2 fields one with @XmlAnyElement(lax=true) List<Object> for storing the elements during the marshaling and another Map<String, Object> with custom serialization for JSON. In addition to this, I got to know that we can use beforeMarshal, afterMarshal, beforeUnmarshal, afterMarshal methods. The name itself suggests what it does.

In my case, I used the beforeMarshal method to add the unknown data from my Map<String, Object> to List<Object> so during the marshaling values from List<Object> will be used. I removed the XMLAdapter.

Also, the afterUnmarshal method to add the read unknown elements from List<Object> to Map<String, Object> so Jackson can utilize it and write to JSON using CustomSearlizer.

Basically, it's a kind of hide-and-show approach. List<Object> will be used during the unmarshalling and marshaling by JAXB/Moxy. Map<String, Object> will be used during the serialization and deserialization by Jackson.

Custome.class with my beforeMarshal and afterUnmarshalling: (It seems bit complex basically it exchanges the data as mentioned above. I will have complex data so I need to recursively loop and arrange. You can make changes according to your need)

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, visible = true, property = "isA")
@JsonInclude(Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "otherElements"})
@XmlAccessorType(XmlAccessType.FIELD)
@Getter
@Setter
@AllArgsConstructor
@ToString
public class Customer {
    @XmlTransient
    private String isA;
    private String name;
    private String age;

    @XmlAnyElement(lax = true)
    @JsonIgnore
    private List<Object> otherElements = new ArrayList<>();


    @JsonIgnore
    @XmlTransient
    private Map<String, Object> userExtensions = new HashMap<>();

    @JsonAnyGetter
    @JsonSerialize(using = CustomExtensionsSerializer.class)
    public Map<String, Object> getUserExtensions() {
        return userExtensions;
    }

    @JsonAnySetter
    public void setUserExtensions(String key, Object value) {
        userExtensions.put(key, value);
    }

    private void beforeMarshal(Marshaller m) throws ParserConfigurationException {
        System.out.println("Before Marshalling User Extension: " + userExtensions);
        ExtensionsModifier extensionsModifier = new ExtensionsModifier();
        otherElements = extensionsModifier.Marshalling(userExtensions);
        System.out.println("Before Marshalling Final Other Elements " + otherElements);
        userExtensions = new HashMap<>();
    }

    private void afterUnmarshal(Unmarshaller m, Object parent) throws ParserConfigurationException {
        System.out.println("After Unmarshalling : " + otherElements);
        ExtensionsModifier extensionsModifier = new ExtensionsModifier();
        userExtensions = extensionsModifier.Unmarshalling(otherElements);
        otherElements = new ArrayList();
    }
}

Then the ExtensionsModifier.class which will be called by beforeMarshal and afterUnmarshalling method:

import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;

import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class ExtensionsModifier {
    private javax.xml.parsers.DocumentBuilderFactory documentFactory;
    private javax.xml.parsers.DocumentBuilder documentBuilder;
    private org.w3c.dom.Document document;

    public ExtensionsModifier() throws ParserConfigurationException {
        documentFactory = DocumentBuilderFactory.newInstance();
        documentBuilder = documentFactory.newDocumentBuilder();
        document = documentBuilder.newDocument();
    }

    public List<Object> Marshalling(Map<String, Object> userExtensions) throws ParserConfigurationException {
        if (userExtensions == null) {
            return null;
        }
        List<Object> tempElement = new ArrayList<>();

        for (Map.Entry<String, Object> property : userExtensions.entrySet()) {
            Element root = document.createElement(property.getKey());
            if (property.getValue() instanceof Map) {
                List<Object> mapElements = Marshalling((Map<String, Object>) property.getValue());
                mapElements.forEach(innerChildren -> {
                    if (innerChildren instanceof Element) {
                        if (((Element) innerChildren).getTextContent() != null) {
                            root.appendChild(document.appendChild((Element) innerChildren));
                        }
                    }
                });
                tempElement.add(root);
            } else if (property.getValue() instanceof String) {
                root.setTextContent(((String) property.getValue()));
                tempElement.add(root);
            } else if (property.getValue() instanceof ArrayList) {
                for (Object dupItems : (ArrayList<Object>) property.getValue()) {
                    if (dupItems instanceof Map) {
                        Element arrayMap = document.createElement(property.getKey());
                        List<Object> arrayMapElements = Marshalling((Map<String, Object>) dupItems);
                        arrayMapElements.forEach(mapChildren -> {
                            if (mapChildren instanceof Element) {
                                if (((Element) mapChildren).getTextContent() != null) {
                                    arrayMap.appendChild(document.appendChild((Element) mapChildren));
                                }
                            }
                        });
                        tempElement.add(arrayMap);
                    } else if (dupItems instanceof String) {
                        Element arrayString = document.createElement(property.getKey());
                        arrayString.setTextContent((String) dupItems);
                        tempElement.add(arrayString);
                    }
                }
            }
        }
        return tempElement;
    }

    public Map<String, Object> Unmarshalling(List<Object> value) {
        if (value == null) {
            return null;
        }
        final Map<String, Object> extensions = new HashMap<>();
        for (Object obj : value) {
            org.w3c.dom.Element element = (org.w3c.dom.Element) obj;
            final NodeList children = element.getChildNodes();

            //System.out.println("Node Name : " + element.getNodeName() + " Value : " + element.getTextContent());
            List<Object> values = (List<Object>) extensions.get(element.getNodeName());

            if (values == null) {
                values = new ArrayList<Object>();
            }

            if (children.getLength() == 1) {
                values.add(element.getTextContent());
                extensions.put(element.getNodeName(), values);
            } else {
                List<Object> child = new ArrayList<>();
                for (int i = 0; i < children.getLength(); i++) {
                    final Node n = children.item(i);
                    if (n.getNodeType() == Node.ELEMENT_NODE) {
                        List<Object> childElements = new ArrayList();
                        childElements.add(n);
                        values.add(Unmarshalling(childElements));
                        child.add(Unmarshalling(childElements));

                    }
                }
                extensions.put(element.getNodeName(), values);
            }
        }
        return extensions;
    }
}

Following is my CustomSearlizer which will be used by Jackson to create JSON:

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializerProvider;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Map;

public class CustomExtensionsSerializer extends JsonSerializer<Map<String, Object>> {

    private static final ObjectMapper mapper = new ObjectMapper();

    @Override
    public void serialize(Map<String, Object> value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
        System.out.println("Custom Json Searlizer: " + value);
        recusiveSerializer(value, gen, serializers);
    }

    public void recusiveSerializer(Map<String, Object> value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
        for (Map.Entry<String, Object> extension : value.entrySet()) {
            if (extension.getValue() instanceof Map) {
                //If instance is MAP then call the recursive method
                recusiveSerializer((Map) extension.getValue(), gen, serializers);
            } else if (extension.getValue() instanceof String) {
                //If instance is String directly add it to the JSON
                gen.writeStringField(extension.getKey(), (String) extension.getValue());
            } else if (extension.getValue() instanceof ArrayList) {
                //If instance if ArrayList then loop over it and add it to the JSON after calling recursive method
                //If size more than 1 add outer elements
                if (((ArrayList<Object>) extension.getValue()).size() > 1) {
                    gen.writeFieldName(extension.getKey());
                    gen.writeStartObject();
                    for (Object dupItems : (ArrayList<Object>) extension.getValue()) {
                        if (dupItems instanceof Map) {
                            recusiveSerializer((Map) dupItems, gen, serializers);
                        } else {
                            gen.writeStringField(extension.getKey(), (String) dupItems);
                        }
                    }
                    gen.writeEndObject();
                } else {
                    for (Object dupItems : (ArrayList<Object>) extension.getValue()) {
                        if (dupItems instanceof Map) {
                            gen.writeFieldName(extension.getKey());
                            gen.writeStartObject();
                            recusiveSerializer((Map) dupItems, gen, serializers);
                            gen.writeEndObject();
                        } else {
                            gen.writeStringField(extension.getKey(), (String) dupItems);
                        }
                    }
                }
            }
        }
    }
}

If I provide input as following XML:

<Customer xmlns:google="https://google.com">
    <name>Rise Against</name>
    <age>2000</age>
    <google:main>
        <google:sub>MyValue</google:sub>
        <google:sub>MyValue</google:sub>
    </google:main>
</Customer>

Then I get the following JSON as output:

{
  "isA" : "Customer",
  "name" : "Rise Against",
  "age" : "2000",
  "google:main" : {
    "google:sub" : "MyValue",
    "google:sub" : "MyValue"
  }
}

Viceversa would also work fine. Hope it's clear if not leave a comment will try to respond.

Liman

Answer 2

I believe this is somewhat related to the issue: @XmlPath(".") conflicts with @XmlAdapter

As per bug ticket:

org.eclipse.persistence.internal.oxm.record.UnmarshalRecordImpl 
public XPathNode getNonAttributeXPathNode(String namespaceURI, String localName, String qName, Attributes attributes) {
...
Line 1279
       if(null == resultNode && null == nonPredicateNode) {
          // ANY MAPPING
          resultNode = xPathNode.getAnyNode();// by default it return the EventAdapter returing a null at this place fix the problem, but i dont know if its the best solution
       }

Anhydrite

Thanks for taking the time and responding. I tried looking into this issue and modified the code accordingly but this did not work for me. Please suggest some workaround. I have requested the work around even on GitHub: github.com/eclipse-ee4j/eclipselink/issues/1181 and github.com/eclipse-ee4j/eclipselink/issues/1172 – Liman

Answer 3

As (just) a workaround, dedicated fields could go to a separate class so it's unmarshal on its own. "Customer" would extend from this class.

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "Person", propOrder = { "name", "age"})
public class Person {

    protected String name;
    protected String age;
//getters and stuff
}

Customer

@XmlRootElement(name = "Customer")
@XmlType(name = "Customer") //, propOrder = { "name", "age", "others" }
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer extends Person{
    @XmlPath(".")
    @XmlJavaTypeAdapter(TestAdapter.class)
    private Map<String, Object> others;
    
}

Custom adapter could not parse elements without namespace or exclude a particular namespace added to Person class.

@Override
public Map<String, Object> unmarshal(Wrapper value) throws Exception {
    System.out.println("INSIDE UNMARSHALLING METHOD TEST");
    final Map<String, Object> others = new HashMap<>();

    for (Object obj : value.getElements()) {
        final Element element = (Element) obj;
        final NodeList children = element.getChildNodes();
        System.out.println(element.getNodeName()+ " children.getLength(): " + children.getLength() + ", ns: " + element.getNamespaceURI());

        if (element.getNamespaceURI() != null) {
            // Check if its direct String value field or complex
            if (children.getLength() == 1) {
                others.put(element.getNodeName(), element.getTextContent());
            } else {
                List<Object> child = new ArrayList<>();
                for (int i = 0; i < children.getLength(); i++) {
                    final Node n = children.item(i);
                    if (n.getNodeType() == Node.ELEMENT_NODE) {
                        Wrapper wrapper = new Wrapper();
                        List<Object> childElements = new ArrayList<Object>();
                        childElements.add(n);
                        wrapper.elements = childElements;
                        child.add(unmarshal(wrapper));
                    }
                }
                others.put(element.getNodeName(), child);
            } 
        }
    }

    return others;
}

Finally, parsing would use DOM object so input is read once

public static void main(String[] args) throws Exception {

    // XML to JSON
    jakarta.xml.bind.JAXBContext jaxbContext = JAXBContext.newInstance(Person.class, Customer.class);
    jakarta.xml.bind.Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
    File initialFile = new File("src/main/resources/Customer.xml");

    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
    dbf.setNamespaceAware(true);
    DocumentBuilder db = dbf.newDocumentBuilder();
    Document doc = db.parse(initialFile);

    final Person person = unmarshaller.unmarshal(doc, Person.class).getValue();
    final Customer customer = unmarshaller.unmarshal(doc, Customer.class).getValue();
    System.out.println("unmarshall:" + customer);

    customer.setName(person.getName());
    customer.setAge(person.getAge());

    final ObjectMapper objectMapper = new ObjectMapper();
    final String jsonEvent = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(customer);
    System.out.println(jsonEvent);

    // JSON to XML
    Marshaller marshaller = jaxbContext.createMarshaller();
    marshaller.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
    marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
    marshaller.marshal(customer, System.out);
}

Nighttime

I really appreciate you taking your time and responding. Thanks a lot for your efforts. I provided a very basic example here. I will try to figure out if it's possible for me to use this in my real application because in the application large file that has many different events. Based on the event I am assigning to a different class so I am using the Jackson while reading JSON along with XmlStreamReader while reading the XML so using with I have to try. Also, I have few standard classes so need to try if I can create a separate class to catch the unmapped elements. – Liman

So I will look into this and try to go by your guidance. If something else clicks to you then please provide that answer as well. Even I am trying few more things like the beforemarshal and beforeunmarshal function to manipulate the data before actually marshaling or unmarshalling. Thanks a lot again for your efforts have a nice day ahead :) – Liman

Thanks a lot again for your answer. I was finally able to find a way without needing to create a separate class for the unknown elements. I have posted the answer here if in case you are interested you can check and provide your suggestion if there is way to make it better: https://mcmap.net/q/282340/-jaxb-moxy-unmarshalling-assigns-all-field-values-to-map-lt-string-object-gt-rather-than-the-specific-field-provided-for-it – Liman

Glad for you :-) – Nighttime