Deserializing polymorphic json classes without type information using json.net
Asked Answered
P

6

116

This Imgur api call returns a list containing both Gallery Image and Gallery Album classes represented in JSON.

I can't see how to deserialize these automatically using Json.NET given that there is no $type property telling the deserializer which class is meant to be represented. There is a property called "IsAlbum" that can be used to differentiate between the two.

This question appears to show one method but it looks like a bit of a hack.

How do I go about deserializing these classes? (using C#, Json.NET).

Sample Data:

Gallery Image

{
    "id": "OUHDm",
    "title": "My most recent drawing. Spent over 100 hours.",
        ...
    "is_album": false
}

Gallery Album

{
    "id": "lDRB2",
    "title": "Imgur Office",
    ...
    "is_album": true,
    "images_count": 3,
    "images": [
        {
            "id": "24nLu",
            ...
            "link": "http://i.imgur.com/24nLu.jpg"
        },
        {
            "id": "Ziz25",
            ...
            "link": "http://i.imgur.com/Ziz25.jpg"
        },
        {
            "id": "9tzW6",
            ...
            "link": "http://i.imgur.com/9tzW6.jpg"
        }
    ]
}
}
Pyrogenous answered 10/10, 2013 at 23:13 Comment(3)
You want to take the Json string and put it into classes? And I'm confused by what you mean by there is no $type property.Proscribe
Yes, I have the json string and want to deserialize to C# classes. Json.NET appears to use a property called $type to draw a distinction between different types held in an array. This data doesnt have that property and just uses the 'IsAlbum' property.Pyrogenous
If you are using System.Text.Json , see is-polymorphic-deserialization-possible-in-system-text-jsonElemental
H
73

Simple with my package that provides the JsonSubTypes attribute that works with Json.NET:

[JsonConverter(typeof(JsonSubtypes), "is_album")]
[JsonSubtypes.KnownSubType(typeof(GalleryAlbum), true)]
[JsonSubtypes.KnownSubType(typeof(GalleryImage), false)]
public abstract class GalleryItem
{
    public string id { get; set; }
    public string title { get; set; }
    public string link { get; set; }
    public bool is_album { get; set; }
}

public class GalleryImage : GalleryItem
{
    // ...
}

public class GalleryAlbum : GalleryItem
{
    public int images_count { get; set; }
    public List<GalleryImage> images { get; set; }
}
Hartzel answered 10/3, 2018 at 21:13 Comment(8)
This should be the top answer. I've spent the better part of a day working on a solution to this problem examining custom JsonConverter classes from dozens of authors. Your nuget package replaced all that effort with three lines of code. Well done sir. Well done.Bund
In Java world, Jackson library provides similar support via @JsonSubTypes attribute. See https://mcmap.net/q/35833/-what-are-jsontypeinfo-and-jsonsubtypes-used-for-in-jackson for another use-case example (also Cage/Animal example by @KonstantinPelepelin in comments).Abacus
This is the simplest answer indeed, but unfortunately it comes at a performance cost. I found out that deserialization is 2-3 times faster with a hand-written converter (as shown in the answer of @BrianRogers)Syringomyelia
Hi @SvenVranckx, feel free to open an issue on github.com/manuc66/JsonSubTypesHartzel
Nice idea, but this didn't work for me unfortunately. I tried the various suggested permutations on the github site, but in the end I needed to go with the handwritten approach.Helicoid
@Frank The status should be fixed, it was due to that dependency being used in tests and licence not found any more by the tool : nuget.org/packages/TaskParallelLibrary . And JsonSubTypes nuget artefact has only one external dependency Newtonsoft.JsonHartzel
@Hartzel I can't do BTC but if you put up an Algorand wallet address I will donate you something during the coming week.Runofthemill
In .NET 7 they added similar functionality out of the box: learn.microsoft.com/en-us/dotnet/standard/serialization/…Copy
C
151

You can do this fairly easily by creating a custom JsonConverter to handle the object instantiation. Assuming you have your classes defined something like this:

public abstract class GalleryItem
{
    public string id { get; set; }
    public string title { get; set; }
    public string link { get; set; }
    public bool is_album { get; set; }
}

public class GalleryImage : GalleryItem
{
    // ...
}

public class GalleryAlbum : GalleryItem
{
    public int images_count { get; set; }
    public List<GalleryImage> images { get; set; }
}

You would create the converter like this:

public class GalleryItemConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return typeof(GalleryItem).IsAssignableFrom(objectType);
    }

    public override object ReadJson(JsonReader reader, 
        Type objectType, object existingValue, JsonSerializer serializer)
    {
        JObject jo = JObject.Load(reader);

        // Using a nullable bool here in case "is_album" is not present on an item
        bool? isAlbum = (bool?)jo["is_album"];

        GalleryItem item;
        if (isAlbum.GetValueOrDefault())
        {
            item = new GalleryAlbum();
        }
        else
        {
            item = new GalleryImage();
        }

        serializer.Populate(jo.CreateReader(), item);

        return item;
    }

    public override bool CanWrite
    {
        get { return false; }
    }

    public override void WriteJson(JsonWriter writer, 
        object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

Here's an example program showing the converter in action:

class Program
{
    static void Main(string[] args)
    {
        string json = @"
        [
            {
                ""id"": ""OUHDm"",
                ""title"": ""My most recent drawing. Spent over 100 hours."",
                ""link"": ""http://i.imgur.com/OUHDm.jpg"",
                ""is_album"": false
            },
            {
                ""id"": ""lDRB2"",
                ""title"": ""Imgur Office"",
                ""link"": ""http://alanbox.imgur.com/a/lDRB2"",
                ""is_album"": true,
                ""images_count"": 3,
                ""images"": [
                    {
                        ""id"": ""24nLu"",
                        ""link"": ""http://i.imgur.com/24nLu.jpg""
                    },
                    {
                        ""id"": ""Ziz25"",
                        ""link"": ""http://i.imgur.com/Ziz25.jpg""
                    },
                    {
                        ""id"": ""9tzW6"",
                        ""link"": ""http://i.imgur.com/9tzW6.jpg""
                    }
                ]
            }
        ]";

        List<GalleryItem> items = 
            JsonConvert.DeserializeObject<List<GalleryItem>>(json, 
                new GalleryItemConverter());

        foreach (GalleryItem item in items)
        {
            Console.WriteLine("id: " + item.id);
            Console.WriteLine("title: " + item.title);
            Console.WriteLine("link: " + item.link);
            if (item.is_album)
            {
                GalleryAlbum album = (GalleryAlbum)item;
                Console.WriteLine("album images (" + album.images_count + "):");
                foreach (GalleryImage image in album.images)
                {
                    Console.WriteLine("    id: " + image.id);
                    Console.WriteLine("    link: " + image.link);
                }
            }
            Console.WriteLine();
        }
    }
}

And here is the output of the above program:

id: OUHDm
title: My most recent drawing. Spent over 100 hours.
link: http://i.imgur.com/OUHDm.jpg

id: lDRB2
title: Imgur Office
link: http://alanbox.imgur.com/a/lDRB2
album images (3):
    id: 24nLu
    link: http://i.imgur.com/24nLu.jpg
    id: Ziz25
    link: http://i.imgur.com/Ziz25.jpg
    id: 9tzW6
    link: http://i.imgur.com/9tzW6.jpg

Fiddle: https://dotnetfiddle.net/1kplME

Corned answered 11/10, 2013 at 0:38 Comment(6)
This does not work if polymorphic objects are recursive, i.e. if an Album may contain other albums. In the converter one should use Serializer.Populate() instead of item.ToObject(). See #29124626Dihedral
For anyone trying out this approach and finding it results in an infinite loop (and eventually a stack overflow) you might want to use the Populate approach instead of ToObject. See the answers to #25404702 and #29124626 . I have an example of the two approaches in a Gist here: gist.github.com/chrisoldwood/b604d69543a5fe5896a94409058c7a95 .Hungnam
I was lost in various answers about 8 hours, they were about CustomCreationConverter. Finally this answer worked and I'm feeling enlightened. My inner object contains it's type as string and I use it to conversion like this. JObject item = JObject.Load(reader); Type type = Type.GetType(item["Type"].Value<string>()); return item.ToObject(type);Gracia
This is working fine for me provided that you do not put the converter attribute on the base class. The converter must be injected into the serializer (via settings etc) and check for only the base type in CanConvert. I am debating using Populate() but I really don't like either method.Geraud
I have fixed the converter to use JsonSerializer.Populate() instead of JObject.ToObject() as was suggested by Ivan and Chris. This will avoid problems with recursive loops and allow the converter to be used successfully with attributes.Corned
Unfortunately this approach doesn't work with [JsonConstructor] - arghWoodhouse
H
73

Simple with my package that provides the JsonSubTypes attribute that works with Json.NET:

[JsonConverter(typeof(JsonSubtypes), "is_album")]
[JsonSubtypes.KnownSubType(typeof(GalleryAlbum), true)]
[JsonSubtypes.KnownSubType(typeof(GalleryImage), false)]
public abstract class GalleryItem
{
    public string id { get; set; }
    public string title { get; set; }
    public string link { get; set; }
    public bool is_album { get; set; }
}

public class GalleryImage : GalleryItem
{
    // ...
}

public class GalleryAlbum : GalleryItem
{
    public int images_count { get; set; }
    public List<GalleryImage> images { get; set; }
}
Hartzel answered 10/3, 2018 at 21:13 Comment(8)
This should be the top answer. I've spent the better part of a day working on a solution to this problem examining custom JsonConverter classes from dozens of authors. Your nuget package replaced all that effort with three lines of code. Well done sir. Well done.Bund
In Java world, Jackson library provides similar support via @JsonSubTypes attribute. See https://mcmap.net/q/35833/-what-are-jsontypeinfo-and-jsonsubtypes-used-for-in-jackson for another use-case example (also Cage/Animal example by @KonstantinPelepelin in comments).Abacus
This is the simplest answer indeed, but unfortunately it comes at a performance cost. I found out that deserialization is 2-3 times faster with a hand-written converter (as shown in the answer of @BrianRogers)Syringomyelia
Hi @SvenVranckx, feel free to open an issue on github.com/manuc66/JsonSubTypesHartzel
Nice idea, but this didn't work for me unfortunately. I tried the various suggested permutations on the github site, but in the end I needed to go with the handwritten approach.Helicoid
@Frank The status should be fixed, it was due to that dependency being used in tests and licence not found any more by the tool : nuget.org/packages/TaskParallelLibrary . And JsonSubTypes nuget artefact has only one external dependency Newtonsoft.JsonHartzel
@Hartzel I can't do BTC but if you put up an Algorand wallet address I will donate you something during the coming week.Runofthemill
In .NET 7 they added similar functionality out of the box: learn.microsoft.com/en-us/dotnet/standard/serialization/…Copy
J
4

Advanced to Brian Rogers answer. And about "use Serializer.Populate() instead of item.ToObject()". If derived types has contstructors or some of their has own customconverter you must use general way for deserialize JSON. So you must leave work for instantiate new object to NewtonJson. This way you can achieve it in you CustomJsonConverter:

public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
    ..... YOU Code For Determine Real Type of Json Record .......

    // 1. Correct ContractResolver for you derived type
    var contract = serializer.ContractResolver.ResolveContract(DeterminedType);
    if (converter != null && !typeDeserializer.Type.IsAbstract && converter.GetType() == GetType())
    {
        contract.Converter = null; // Clean Wrong Converter grabbed by DefaultContractResolver from you base class for derived class
    }

    // Deserialize in general way           
    var jTokenReader = new JTokenReader(jObject);
    var result = serializer.Deserialize(jTokenReader, DeterminedType);

    return (result);
}

This work if you have recursion of objects.

Jasisa answered 15/2, 2019 at 13:24 Comment(4)
This code is not safe in mutithreaded contexts, because the default contract resolver caches contracts. Setting options on the contract (like its converter) will cause it to behave differently on concurrent, and even subsequent, calls.Argentiferous
serializer.ContractResolver.ResolveContract(DeterminedType) - return already cached contract. So contract.Converter = null; change cached object. It only change reference on cached object, and it is thread safe.Daisie
This is actually a very good answer, and it keeps everything very simple. If you use Serializer.Populate(), you need to create the object yourself, and all logic for creating the objects (e.g. JsonConstructor attributes) will be ignored.Paterson
This worked for me when my base type BaseDto had [JsonConverter( typeof(MyConverter) )] however when I removed the attribute (but still specified MyConverter in JsonSerializer.Converters) it stopped working because contract.Converter was always null. I'm unsure how I should indicate to JsonSerializer that subclasses of Dto should also use the same converter, hmm. (Unfortunately JsonConverter<T> makes the CanConvert method sealed argh!)Woodhouse
G
1

I'm only posting this to clear up some of the confusion. If you are working with a predefined format and need to deserialize it, this is what I found worked best and demonstrates the mechanics so that others can tweak it as needed.

public class BaseClassConverter : JsonConverter
    {
        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            var j = JObject.Load(reader);
            var retval = BaseClass.From(j, serializer);
            return retval;
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            serializer.Serialize(writer, value);
        }

        public override bool CanConvert(Type objectType)
        {
            // important - do not cause subclasses to go through this converter
            return objectType == typeof(BaseClass);
        }
    }

    // important to not use attribute otherwise you'll infinite loop
    public abstract class BaseClass
    {
        internal static Type[] Types = new Type[] {
            typeof(Subclass1),
            typeof(Subclass2),
            typeof(Subclass3)
        };

        internal static Dictionary<string, Type> TypesByName = Types.ToDictionary(t => t.Name.Split('.').Last());

        // type property based off of class name
        [JsonProperty(PropertyName = "type", Required = Required.Always)]
        public string JsonObjectType { get { return this.GetType().Name.Split('.').Last(); } set { } }

        // convenience method to deserialize a JObject
        public static new BaseClass From(JObject obj, JsonSerializer serializer)
        {
            // this is our object type property
            var str = (string)obj["type"];

            // we map using a dictionary, but you can do whatever you want
            var type = TypesByName[str];

            // important to pass serializer (and its settings) along
            return obj.ToObject(type, serializer) as BaseClass;
        }


        // convenience method for deserialization
        public static BaseClass Deserialize(JsonReader reader)
        {
            JsonSerializer ser = new JsonSerializer();
            // important to add converter here
            ser.Converters.Add(new BaseClassConverter());

            return ser.Deserialize<BaseClass>(reader);
        }
    }
Geraud answered 18/9, 2017 at 20:42 Comment(2)
How do you use this when using implicit conversion without using the [JsonConverter()] attribute (which is commented as "important")? Eg: deserializing via [FromBody] attribute?Baalman
I assume you can simply edit the global JsonFormatter's settings to include this converter. See #41630023Geraud
A
1

Following implementation should let you de-serialize without changing the way you have designed your classes and by using a field other than $type to decide what to de-serialize it into.

public class GalleryImageConverter : JsonConverter
{   
    public override bool CanConvert(Type objectType)
    {
        return (objectType == typeof(GalleryImage) || objectType == typeof(GalleryAlbum));
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        try
        {
            if (!CanConvert(objectType))
                throw new InvalidDataException("Invalid type of object");
            JObject jo = JObject.Load(reader);
            // following is to avoid use of magic strings
            var isAlbumPropertyName = ((MemberExpression)((Expression<Func<GalleryImage, bool>>)(s => s.is_album)).Body).Member.Name;
            JToken jt;
            if (!jo.TryGetValue(isAlbumPropertyName, StringComparison.InvariantCultureIgnoreCase, out jt))
            {
                return jo.ToObject<GalleryImage>();
            }
            var propValue = jt.Value<bool>();
            if(propValue) {
                resultType = typeof(GalleryAlbum);
            }
            else{
                resultType = typeof(GalleryImage);
            }
            var resultObject = Convert.ChangeType(Activator.CreateInstance(resultType), resultType);
            var objectProperties=resultType.GetProperties();
            foreach (var objectProperty in objectProperties)
            {
                var propType = objectProperty.PropertyType;
                var propName = objectProperty.Name;
                var token = jo.GetValue(propName, StringComparison.InvariantCultureIgnoreCase);
                if (token != null)
                {
                    objectProperty.SetValue(resultObject,token.ToObject(propType)?? objectProperty.GetValue(resultObject));
                }
            }
            return resultObject;
        }
        catch (Exception ex)
        {
            throw;
        }
    }

    public override bool CanWrite
    {
        get { return false; }
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}
Achaea answered 16/1, 2018 at 18:31 Comment(0)
W
0

@ИгорьОрлов's answer works for when you have types that can only be instantiated directly by JSON.net (due to [JsonConstructor] and/or use of [JsonProperty] directly on constructor parameters. However overwriting contract.Converter = null does not work when JSON.net has already cached the converter to use.

(This wouldn't be an issue if JSON.NET used immutable types to indicate when data and configuration is no-longer mutable, le sigh)

In my case, I did this:

  1. Implemented a custom JsonConverter<T> (where T is my DTO's base class).
  2. Defined a DefaultContractResolver subclass that overrides ResolveContractConverter to return my custom JsonConverter for only the base class.

In detail, and by example:

Supposing I have these immutable DTOs that represent a remote file-system (so there' DirectoryDto and FileDto which both inherit FileSystemDto, just like how DirectoryInfo and FileInfo derive from System.IO.FileSystemInfo):

public enum DtoKind
{
    None = 0,
    File,
    Directory
}

public abstract class FileSystemDto
{
    protected FileSystemDto( String name, DtoKind kind )
    {
        this.Name = name ?? throw new ArgumentNullException(nameof(name));
        this.Kind = kind;
    }

    [JsonProperty( "name" )]
    public String Name { get; }

    [JsonProperty( "kind" )]
    public String Kind { get; }
}

public class FileDto : FileSystemDto
{
    [JsonConstructor]
    public FileDto(
        [JsonProperty("name"  )] String  name,
        [JsonProperty("length")] Int64   length,
        [JsonProperty("kind")  ] DtoKind kind
    )
        : base( name: name, kind: kind )
    {
        if( kind != DtoKind.File ) throw new InvalidOperationException( "blargh" );
        this.Length = length;
    }

    [JsonProperty( "length" )]
    public Int64 Length { get; }
}

public class DirectoryDto : FileSystemDto
{
    [JsonConstructor]
    public FileDto(
        [JsonProperty("name")] String  name,
        [JsonProperty("kind")] DtoKind kind
    )
        : base( name: name, kind: kind )
    {
        if( kind != DtoKind.Directory ) throw new InvalidOperationException( "blargh" );
    }
}

Supposing I have a JSON array of FileSystemDto:

[
    { "name": "foo.txt", "kind": "File", "length": 12345 },
    { "name": "bar.txt", "kind": "File", "length": 12345 },
    { "name": "subdir", "kind": "Directory" },
]

I want Json.net to deserialize this to List<FileSystemDto>...

So define a subclass of DefaultContractResolver (or if you already have a resolver implementation then subclass (or compose) that) and override ResolveContractConverter:

public class MyContractResolver : DefaultContractResolver
{
    protected override JsonConverter? ResolveContractConverter( Type objectType )
    {
        if( objectType == typeof(FileSystemDto) )
        {
            return MyJsonConverter.Instance;
        }
        else if( objectType == typeof(FileDto ) )
        {
            // use default
        }
        else if( objectType == typeof(DirectoryDto) )
        {
            // use default
        }

        return base.ResolveContractConverter( objectType );
    }
}

Then implement MyJsonConverter:

public class MyJsonConverter : JsonConverter<FileSystemDto>
{
    public static MyJsonConverter Instance { get; } = new MyJsonConverter();

    private MyJsonConverter() {}

    // TODO: Override `CanWrite => false` and `WriteJson { throw; }` if you like.

    public override FileSystemDto? ReadJson( JsonReader reader, Type objectType, FileSystemDto? existingValue, Boolean hasExistingValue, JsonSerializer serializer )
    {
        if( reader.TokenType == JsonToken.Null ) return null;

        if( objectType == typeof(FileSystemDto) )
        {
            JObject jsonObject = JObject.Load( reader );
            if( jsonObject.Property( "kind" )?.Value is JValue jv && jv.Value is String kind )
            {
                if( kind == "File" )
                {
                    return jsonObject.ToObject<FileDto>( serializer );
                }
                else if( kind == "Directory" )
                {
                    return jsonObject.ToObject<DirectoryDto>( serializer );
                }
            }
        }

        return null; // or throw, depending on your strictness.
    }
}

Then, to deserialize, use a JsonSerializer instance with the ContractResolver set correctly, for example:

public static IReadOnlyList<FileSystemDto> DeserializeFileSystemJsonArray( String json )
{
    JsonSerializer jss = new JsonSerializer()
    {
        ContractResolver = new KuduDtoContractResolver()
    };

    using( StringReader strRdr = new StringReader( json ) )
    using( JsonTextReader jsonRdr = new JsonTextReader( strRdr ) )
    {
        List<FileSystemDto>? list = jss.Deserialize< List<FileSystemDto> >( jsonRdr );
        // TODO: Throw if `list` is null.
        return list;
    }
}
Woodhouse answered 17/8, 2021 at 23:31 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.