Overlay data from JSON string to existing object instance

Question

I want to deserialize a JSON string which does not necessarily contain data for every member, e.g:

public class MyStructure
{
   public string Field1;
   public string Field2;
}

Suppose I have an instance:

Field1: "data1"
Field2: "data2"

and I deserialize a string:

{ "Field1": "newdata1" }

The result should be

Field1: "newdata1"
Field2: "data2"

Framework JavascriptSerializer and JSON.NET both return new objects in their deserialize methods, so the only way I can think of doing this directly would be to compare the deserialized object with the existing one using reflection which seems like a lot of unnecessary overhead. Ideally, some software would have a method in which I passed an existing instance of an object, and only those members which existed in the string would get updated. The point here is that I would like to be able to pass only data which has changed to the server, and update an existing object.

Is this possible using either of these tools, and if not, any suggestions on how to approach the problem?

Chaulmoogra

When people talk about "deserializing," they typically expect a new object. Are you saying that you want behavior similar to MVC's UpdateModel method? You provide an object, and the framework sets whatever values it finds in the input string? – Bixby

... but not using MVC :) – Chaulmoogra

@jamietre: What @StriplingWarror said is true, this is not deserialization from a .NET perspective, it's overlaying data on an existing instance. – Brasca

I never really gave it much thought, I think of "serializing" and "deserializing" as the process of mapping a data structure to a portable format, and vice versa. The distinction about whether one creates a new instance, or maps to an existing one, seems minor. I would like to use the correct terminology, though, but "overlaying data" does not seem capture that I'm talking about data coming from a JSON string, as opposed to data already in the native format or another existing object. – Chaulmoogra

Answer 1

After poking around the source code (so much easier than reading the documentation, eh?) JSON.NET does exactly what I want already:

JsonConvert.PopulateObject(string, object)

See Json.NET: Populate an Object

Chaulmoogra

For me new JsonSerializerSettings{NullValueHandling = NullValueHandling.Ignore} as third parameter was truly helpful. – Concentrate

Answer 2

Realize - JsonConvert.PopulateObject(string,object) will NOT work for collections.

Even with PreserveReferencesHandling = Objects/Arrays/All and an IReferenceResolver. JSON.NET will not update items in collections. Instead, it will duplicate your collection items.

JSON.NET only uses its ("ref") Preserve Reference identifiers to reuse references read within the serialized JSON. JSON.NET will not reuse instances in existing nested object graphs. We attempted by adding an ID property to all our objects, but JSON.NET IReferenceResolver does not provide the facilities to find & match existing references within collections.

Our solution will be to deserialize JSON into a new object instance and map properties across the 2 instances using either Fasterflect or AutoMapper.

Frenchify

Answer 3

Note that JsonConvert.PopulateObject

JsonConvert.PopulateObject(json, item, new JsonSerializerSettings());

Simply calls jsonSerializer.Populate (see here)

        string json = "{ 'someJson':true }";

        var jsonSerializer = new JsonSerializer();

        jsonSerializer.Populate(new StringReader(json), item);

So if you need to repeatedly convert a thousand objects, you may get better performance this route, so that a new JsonSerializer is not instantiated every time.

Babylonian

I don't disagree but our services generally create a single copy of the serializer settings at startup and reuse for the life of the app. The downside is something could always change a setting but we've lived with that risk for multiple years now. Great Q & A's here for something I was looking for. – Marshall

@NoRefundsNoReturns just send in your pre-instantiated settings instead of creating a new instance eh? – Babylonian

Answer 4

I came across this post, and thought I would share my solution for dealing with arrays, as I couldn't find a fully worked up example anywhere. In order for this sample to work, the target array must implement IEnumerable and IList, and the target array objects must implement IEquatable(Of JToken). The implementation of IEquatable(Of JToken) is where you put your logic to determine whether the deserializer should act on an existing item or create a new one. The example also removes any items from the target that are not in the json. I haven't added a disposal check on the removed items, but trivial to do.

The new PopulateObject Call:

Private Sub PopulateObject(value As String, target As Object)

    'set up default converter
    Dim converter As ReconcileEnumerationConverter = New ReconcileEnumerationConverter

    JsonConvert.DefaultSettings = Function()
                                      Return New JsonSerializerSettings With {.Converters = {converter}}
                                  End Function

    'for some reason populate object won't call converter on root
    'so force the issue if our root is an array
    If converter.CanConvert(target.GetType) Then
        Dim array As JArray = JArray.Parse(value)
        converter.ReadJson(array.CreateReader, target.GetType, target, Nothing)
    Else
        JsonConvert.PopulateObject(value, target)
    End If

End Sub

The converter:

Public Class ReconcileEnumerationConverter : Inherits JsonConverter

    Public Overrides Function CanConvert(objectType As Type) As Boolean
        'check to ensure our target type has the necessary interfaces
        Return GetType(IList).IsAssignableFrom(objectType) AndAlso GetType(IEnumerable(Of IEquatable(Of JToken))).IsAssignableFrom(objectType)
    End Function

    Public Overrides ReadOnly Property CanWrite As Boolean
        Get
            Return False
        End Get
    End Property

    Public Overrides Function ReadJson(reader As JsonReader, objectType As Type, existingValue As Object, serializer As JsonSerializer) As Object

        Dim array As JArray = JArray.ReadFrom(reader)

        'cast the existing items
        Dim existingItems As IEnumerable(Of IEquatable(Of JToken)) = CType(existingValue, IEnumerable(Of IEquatable(Of JToken)))
        'copy the existing items for reconcilliation (removal) purposes
        Dim unvisitedItems As IList = existingItems.ToList 'start with full list, and remove as we go
        'iterate each item in the json array
        For Each j As JToken In array.Children
            'look for existing
            Dim existingitem As Object = existingItems.FirstOrDefault(Function(x) x.Equals(j))
            If existingitem IsNot Nothing Then 'found an existing item, update it
                JsonSerializer.CreateDefault.Populate(j.CreateReader, existingitem)
                unvisitedItems.Remove(existingitem)
            Else 'create a new one
                Dim newItem As Object = JsonSerializer.CreateDefault.Deserialize(j.CreateReader)
                CType(existingItems, IList).Add(newItem)
            End If
        Next
        'remove any items not visited
        For Each item As Object In unvisitedItems
            CType(existingItems, IList).Remove(item)
        Next
        Return existingItems

    End Function

    Public Overrides Sub WriteJson(writer As JsonWriter, value As Object, serializer As JsonSerializer)
        Throw New NotImplementedException
    End Sub

End Class

And a sample implementation of IEquatable(of JToken), keyed on an integer 'Id' field:

Public Shadows Function Equals(other As JToken) As Boolean Implements IEquatable(Of JToken).Equals
    Dim idProperty As JProperty = other.Children.FirstOrDefault(Function(x) CType(x, JProperty).Name = "Id")
    If idProperty IsNot Nothing AndAlso CType(idProperty.Value, JValue).Value = Id Then
        Return True
    Else
        Return False
    End If
End Function

Hienhieracosphinx

Thanks for the answer with full implementation! Personally, if doing an update/replacement overlay I would default to NOT removing preexisting items; but that change is just removing the unvisitedItems handling plus a tweak to make the IEnumerable/IEquatable requirement optional. BTW: The (non-generic) IList requirement can be downgraded to an ICollection requirement. – Skysweeper

FWIW that last code snippet is much simpler in c#: => (other.Children.OfType<JProperty>().FirstOrDefault(jp => jp.Name == "Id")?.Value as JValue)?.Value == Id – Onfroi