How to serialize/deserialize a custom collection with additional properties using Json.Net

Question

I have a custom collection (implements IList) which has some custom properties as shown below:

class FooCollection : IList<Foo> {

    private List<Foo> _foos = new List<Foo>();
    public string Bar { get; set; }        

    //Implement IList, ICollection and IEnumerable members...

}

When I serialize, I use the following code:

JsonSerializerSettings jss = new JsonSerializerSettings() {
    TypeNameHandling = TypeNameHandling.Auto
};
string serializedCollection = JsonConvert.SerializeObject( value , jss );

It serializes and deserializes all the collection items properly; however, any extra properties in the FooCollection class are not taken into account.

Is there anyway to include them in the serialization?

Messapian

Answer 1

The problem is the following: when an object implements IEnumerable, JSON.net identifies it as an array of values and serializes it following the array Json syntax (that does not include properties), e.g. :

 [ {"FooProperty" : 123}, {"FooProperty" : 456}, {"FooProperty" : 789}]

If you want to serialize it keeping the properties, you need to handle the serialization of that object by hand by defining a custom JsonConverter :

// intermediate class that can be serialized by JSON.net
// and contains the same data as FooCollection
class FooCollectionSurrogate
{
    // the collection of foo elements
    public List<Foo> Collection { get; set; }
    // the properties of FooCollection to serialize
    public string Bar { get; set; }
}

public class FooCollectionConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(FooCollection);
    }

    public override object ReadJson(
        JsonReader reader, Type objectType, 
        object existingValue, JsonSerializer serializer)
    {
        // N.B. null handling is missing
        var surrogate = serializer.Deserialize<FooCollectionSurrogate>(reader);
        var fooElements = surrogate.Collection;
        var fooColl = new FooCollection { Bar = surrogate.Bar };
        foreach (var el in fooElements)
            fooColl.Add(el);
        return fooColl;
    }

    public override void WriteJson(JsonWriter writer, object value, 
                                   JsonSerializer serializer)
    {
        // N.B. null handling is missing
        var fooColl = (FooCollection)value;
        // create the surrogate and serialize it instead 
        // of the collection itself
        var surrogate = new FooCollectionSurrogate() 
        { 
            Collection = fooColl.ToList(), 
            Bar = fooColl.Bar 
        };
        serializer.Serialize(writer, surrogate);
    }
}

Then use it as follows:

var ss = JsonConvert.SerializeObject(collection, new FooCollectionConverter());

var obj = JsonConvert.DeserializeObject<FooCollection>(ss, new FooCollectionConverter());

Hillie

This helped me, however JSON.NET did not like it when I used Foo[] as the type in the surrogate - I had to use an IEnumerable<Foo> – Traceetracer

Mmh... strange, what version of json.net are you using ? – Hillie

Not sure, can't check at this time, but the one from nugget targeting .NET 4.0 – Traceetracer

I've just checked and it works fine with both version 4.5.11.15520 and 5.0.8.16617... I suspect you're doing something slightly different from the code above... – Hillie

Well, I got an exception saying something along the lines of "cannot preserve reference to an array or read-only list, ... stuff I don't remember". Regardless, using IEnumerable (List implementation) works for me. :) – Traceetracer

@Jeff: Ah you have activated the PreserveReferencesHandling... so yes, there's a problem with that mode and arrays, so of course if you just change the collection to List or IEnumerable it works ;) – Hillie

Why is there a problem with arrays? They are passed by reference, aren't they? – Traceetracer

I don't know exactly the reason, but readonly lists and arrays do not work with preserve reference handling on... (there's also a JSON.net test that includes this case) – Hillie

Well, in that case, you might want to add that to your answer for future readers. :) – Traceetracer

@Jeff: I've replaced the array with a list, so people won't have a crash in case of preserve-reference = true – Hillie

Answer 2

Personally I like to avoid writing custom JsonConverters where possible, and instead make use of the various JSON attributes which were designed for this purpose. You can simply decorate FooCollection with JsonObjectAttribute, which forces serialization as a JSON object rather than an array. You'd have to decorate the Count and IsReadOnly properties with JsonIgnore to prevent them from showing up in the output. If you want to keep _foos a private field, you would also have to decorate it with JsonProperty.

[JsonObject]
class FooCollection : IList<Foo> {
    [JsonProperty]
    private List<Foo> _foos = new List<Foo>();
    public string Bar { get; set; }  

    // IList implementation
    [JsonIgnore]
    public int Count { ... }
    [JsonIgnore]
    public bool IsReadOnly { ... }
}

Serializing yields something like the following:

{
  "_foos": [
    "foo1",
    "foo2"
  ],
  "Bar": "bar"
}

Obviously this only works if you are able to change the definition of FooCollection in order to add those attributes, otherwise you have to go the way of custom converters.

Traceetracer

Since I'm working with c# models that are persisted json documents, this was the best option for me. I'm entirely fine with decorating my model with information that mirrors its physical persistence. As this case the documents are 100% coupled to the json. – Winniewinnifred

I think you can avoid the JsonIgnore by implementing the interface explicitly. – Sweyn

Today I learnt that using explicit interfaces it will not serialise those members. Problem is I don't know why? Whats the story behind this Little Endian – Numismatist

Answer 3

If you also want to keep the contents of the List or collection itself, You should consider exposing a property to return the list. It has to be wrapped to prevent cyclic issues while serializing:

Note: This solution supports both serializing/deserializing.

[JsonObject]
public class FooCollection : List<int>
{
    public string Bar { get; set; } = "Bar";

    [JsonProperty]
    ICollection<int> Items => new _<int>(this);
}

internal class _<T> : ICollection<T>
{
    public _(ICollection<T> collection) => inner = collection;
    private ICollection<T> inner;
    int ICollection<T>.Count => inner.Count;
    bool ICollection<T>.IsReadOnly => inner.IsReadOnly;
    void ICollection<T>.Add(T item) => inner.Add(item);
    void ICollection<T>.Clear() => inner.Clear();
    bool ICollection<T>.Contains(T item) => inner.Contains(item);
    void ICollection<T>.CopyTo(T[] array, int arrayIndex) => inner.CopyTo(array, arrayIndex);
    IEnumerator<T> IEnumerable<T>.GetEnumerator() => inner.GetEnumerator();
    bool ICollection<T>.Remove(T item) => inner.Remove(item);
    IEnumerator IEnumerable.GetEnumerator() => inner.GetEnumerator();
}

new FooCollection { 1, 2, 3, 4, 4 } =>

{
  "bar": "Bar",
  "items": [
    1,
    2,
    3
  ],
  "capacity": 4,
  "count": 3
}

new FooCollection { 1, 2, 3 }.ToArray() => new []{1, 2, 3}.ToArray()

Exocentric

Answer 4

If you don't want to write custom JsonConverter or use JSON attributes (JsonObjectAttribute), you could use following extension method:

public static string ToFooJson<T>(this FooCollection fooCollection)
{
     return JsonConvert.SerializeObject(new
     {
         Bar = fooCollection.Bar,
         Collection = fooCollection
     });
}

Scourings

Answer 5

Does inheriting from List work?

class FooCollection : List<Foo>, IList<Foo>
{
    public string Bar { get; set; }        
    //Implement IList, ICollection and IEnumerable members...
}

Hower

It'd be surprising, however, unless ABSOLUTELY necessary I'll go that route, ideally I want to keep inheritance the same. – Messapian

Currently you're trying to serialize a private property - by inheriting you become the object you're asking to be serialized (this is obviously not the only way, but could well be the easiest) – Hower

As @Hillie points out, this won't work because JSON.net uses the collection serializer, which doesn't check for additional properties. However, I must agree with qujck that if your class IS A List<> then it is best to inherit directly from List<> rather than have to implement the whole IList<> interface. The only exception is if it already has to implement some other fat base class. – Collaboration

Answer 6

I was testing something similar recently and came up with the following sample (very similar to @Ahmed Alejo's answer, but without requiring public properties) which doesn't require a custom serializer:

void Main()
{
    var test = new Container();
    test.Children.Add("Item 1");
    test.Children.Add("Item 2");
    test.Children.Add("Item 3");
    
    Console.WriteLine(JsonConvert.SerializeObject(test));
}

class Container {
    public CustomList Children { get; set; } = new CustomList() { Name = Guid.NewGuid().ToString() };
}

[JsonObject(MemberSerialization = MemberSerialization.Fields)]
class CustomList : IList<string>
{
    private List<string> items = new List<string>();
    private string name;

    public string this[int index] { get => items[index]; set => items[index] = value; }

    public string Name { get => name; set { name = value; } }

    public int Count => items.Count;

    public bool IsReadOnly => false;

    public void Add(string item)
    {
        items.Add(item);
    }

    public void Clear()
    {
        items.Clear();
    }

    public bool Contains(string item) => items.Contains(item);

    public void CopyTo(string[] array, int arrayIndex)
    {
        items.CopyTo(array, arrayIndex);
    }

    public IEnumerator<string> GetEnumerator() => items.GetEnumerator();

    public int IndexOf(string item) => items.IndexOf(item);

    public void Insert(int index, string item)
    {
        items.Insert(index, item);
    }

    public bool Remove(string item) => items.Remove(item);

    public void RemoveAt(int index)
    {
        items.RemoveAt(index);
    }

    IEnumerator IEnumerable.GetEnumerator() => ((IEnumerable)items).GetEnumerator();
}

The key is in adding the explicit JsonObjectAttribute to the class, which will treat it like an object instead of like an array (which can also be explicitly denoted using the JsonArrayAttribute). The (beautified) output of the above test is as follows:

{
  "Children": {
    "items": [
      "Item 1",
      "Item 2",
      "Item 3"
    ],
    "name": "ff8768f7-5efb-4622-b7e2-f1472e80991c"
  }
}

Helaina