JSON deserialization - Map array indices to properties with JSON.NET

Question

I want to deserialize a 2-dimensional array to a collection of .net objects. The reason is, array syntax will be easier for my user to work with in an input file. So I just want to map the indices of the arrays to specific properties of my target type.

E.G. With:

[
     ["John", "Smith", "23"],
     ["Paula", "Martin", "54]
]

I would get two instances of a Person:

public class Person {
    public string First {get;set;}
    public string Last {get;set;}
    public string Age {get;set;}
}

where index 0 of an inner array maps to First, index 1 maps to Last, and index 2 maps to Age;

Is there a way to extend Json.NET so that I can do the mapping during deserialization so the implementation details are hidden? I have been playing around with a custom JsonConverter but I haven't found much info on how to use it.

Edit: Specifically, I'm not sure if JsonConverter is the right thing to use, and I'm having trouble figuring out how to implement CanConvert and how to use the parameters passed to the ReadJson method.

Bowerbird

Looks like something I'd try to do - but I would suggest that in the end, youll just end up using standard json. The fieldnames must be repeated for each 'Person', but it's a small price. – Grieve

Answer 1

You can do this with a JsonConverter. A simple converter for this purpose would be:

public class PersonConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(Person);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return null;
        var array = JArray.Load(reader);
        var person = (existingValue as Person ?? new Person());
        person.First = (string)array.ElementAtOrDefault(0);
        person.Last = (string)array.ElementAtOrDefault(1);
        person.Age = (string)array.ElementAtOrDefault(2);
        return person;
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var person = (Person)value;
        serializer.Serialize(writer, new[] { person.First, person.Last, person.Age });
    }
}

If the specific properties have non-primitive types, you can use JToken.ToObject<T>(JsonSerializer) to deserialize them to the required type:

person.First = array.ElementAtOrDefault(0)?.ToObject<string>(serializer);

Then you can apply it to your class:

[JsonConverter(typeof(PersonConverter))]
public class Person
{
    public string First { get; set; }
    public string Last { get; set; }
    public string Age { get; set; }
}

Or use it in settings:

var settings = new JsonSerializerSettings { Converters = new [] { new PersonConverter() } };
var list = JsonConvert.DeserializeObject<List<Person>>(json, settings);

Starspangled

This seems like it's on the right track, but it appears to only handle a single Person array, not an array of Person arrays. I'll mark as an answer for confirming a JsonConverter can do it. See my answer for the implementation I came up with. Please let me know if you see any problems with it. – Bowerbird

Answer 2

For a general purpose converter, here's an implementation that uses attributes to associate the properties:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, AllowMultiple = false)]
public class JsonArrayIndexAttribute : Attribute
{
    public JsonArrayIndexAttribute(int index)
    {
        Index = index;
    }
    
    public int Index { get; }
}

public class JsonArrayConverter<T> : JsonConverter<T>
{
    public override T? ReadJson(JsonReader reader, Type objectType, T? existingValue, bool hasExistingValue, JsonSerializer serializer)
    {
        switch (reader.TokenType)
        {
            case JsonToken.StartArray:
                var members = GetIndexedMembers(objectType);
                var arr = JArray.ReadFrom(reader);
                var fromArray = hasExistingValue ? existingValue! : Activator.CreateInstance<T>()!;
                foreach (var (member, value) in members.Zip(arr))
                {
                    member.setter?.Invoke(fromArray, Convert.ChangeType(value, member.type));
                }
                return fromArray;
            case JsonToken.StartObject:
                var fromObject = hasExistingValue ? existingValue! : Activator.CreateInstance<T>()!;
                serializer.Populate(reader, fromObject);
                return fromObject;
            case JsonToken.Null:
                return default;
            case JsonToken.Undefined:
                return hasExistingValue ? existingValue : default;
            default:
                throw new JsonSerializationException($"Unexpected TokenType: {reader.TokenType}");
        }
    }

    public override void WriteJson(JsonWriter writer, T? value, JsonSerializer serializer)
    {
        if (value == null)
        {
            writer.WriteNull();
            return;
        }
        var members = GetIndexedMembers(value.GetType());
        writer.WriteStartArray();
        foreach (var member in members)
        {
            writer.WriteValue(member.getter?.Invoke(value));
        }
        writer.WriteEndArray();
    }
    
    private IEnumerable<(Type? type, Func<object, object?>? getter, Action<object, object?>? setter)> GetIndexedMembers(Type type)
    {
        var indexedMembers =
            (from member in type.GetMembers()
            let index = (JsonArrayIndexAttribute?)member.GetCustomAttribute(typeof(JsonArrayIndexAttribute))
            where index != null
            select (member, index.Index))
            .ToLookup(x => x.Index, x => x.member);

        return
            (from i in Enumerable.Range(0, indexedMembers.Max(x => x.Key) + 1)
            from m in indexedMembers[i].TakeLast(1).DefaultIfEmpty()
            select CreateAccessors(m));
            
        (Type, Func<object, object?>?, Action<object, object?>?) CreateAccessors(MemberInfo m) => m switch
        {
            PropertyInfo p => (p.PropertyType, obj => p.GetValue(obj), (obj, value) => p.SetValue(obj, value)),
            FieldInfo f => (f.FieldType, obj => f.GetValue(obj), (obj, value) => f.SetValue(obj, value)),
            _ => default,
        };
    }
}

Then to use it in your case:

[JsonConverter(typeof(JsonArrayConverter<Person>))]
public class Person
{
    [JsonArrayIndex(0)]
    public string First { get; set; } = default!;
    
    [JsonArrayIndex(1)]
    public string Last { get; set; } = default!;
    
    [JsonArrayIndex(2)]
    public int Age { get; set; }
}

Mnemonic

This should be added into System.Text.Json !!!!!!!! – Carew

I've adapted your solution to System.Text.Json in my own answer below. – Carew

I don't think it would be as useful to be added to the base framework. Serializing an object as an array is not a common thing. It might be more useful generally for types that can be deconstructed to tuples however. – Mnemonic

Javascript "tuples" i.e. arrays of fixed structure are not a rare thing, and definitely worth creating a general case for. – Carew

Answer 3

    public override bool CanConvert(Type objectType)
    {
        if (objectType == typeof(List<Person>)) return true;

        return false;
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        List<Person> persons = new List<Person>();

        JArray personsArray = (JArray)serializer.Deserialize(reader);

        foreach (var personArray in personsArray.Children<JArray>())
        {
            persons.Add(new Person() { 
                         First = personArray[0].Value<string>(),
                         Last = personArray[1].Value<string>(),
                         Age = personArray[2].Value<string>()
                        });
        }

        return persons;
    }

Bowerbird

Answer 4

I've adapted the general solution by Jeff Mercado (in another answer to this question) which was written for Newtonsoft into a System.Text.Json compatible solution:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field)]
public class JsonArrayIndexAttribute(int index) : Attribute {
    public int Index { get; } = index;
}

public class JsonArrayConverter<T> : JsonConverter<T> {
    private static readonly IndexedPropertyDescriptor NoOperation =
        new(typeof(object), _ => null, (_, _) => { });

    public override T Read(
        ref Utf8JsonReader reader,
        Type typeToConvert,
        JsonSerializerOptions options
    ) {
        var returnedObject =
            Activator.CreateInstance(typeToConvert) ??
            throw new JsonException($"Couldn't create object of type {typeToConvert}");

        var members = GetIndexedMembers(typeToConvert);
        foreach (var member in members) {
            reader.Read();

            if (reader.TokenType == JsonTokenType.EndArray) {
                // Return early if array ends before all members have been assigned
                return (T)returnedObject;
            }

            var parsedValue = JsonSerializer.Deserialize(
                reader: ref reader,
                returnType: member.Type,
                options: options);

            member.Setter(returnedObject, parsedValue);
        }

        // Eat up the rest of the elements, if any.
        while (true) {
            reader.Read();
            if (reader.TokenType == JsonTokenType.EndArray) {
                break;
            }

            _ = JsonSerializer.Deserialize(
                reader: ref reader,
                returnType: typeof(object),
                options: options);
        }

        return (T)returnedObject;
    }

    public override void Write(
        Utf8JsonWriter writer,
        T value,
        JsonSerializerOptions options
    ) {
        if (value is null) {
            writer.WriteNullValue();
            return;
        }

        writer.WriteStartArray();
        var members = GetIndexedMembers(typeof(T));
        foreach (var member in members) {
            var sourceValue = member.Getter(value);
            JsonSerializer.Serialize(
                writer: writer,
                value: sourceValue,
                inputType: member.Type,
                options: options);
        }

        writer.WriteEndArray();
    }

    private static IEnumerable<IndexedPropertyDescriptor>
        GetIndexedMembers(Type type) {
        var indexedMembers = (
                from member in type.GetMembers()
                let index = (JsonArrayIndexAttribute?)member
                    .GetCustomAttribute(typeof(JsonArrayIndexAttribute))
                where index != null
                select (member, index.Index))
            .ToLookup(x => x.Index, x => x.member);

        var max = indexedMembers.Count == 0 
            ? 0 
            : indexedMembers.Max(x => x.Key) + 1;

        return
            from i in Enumerable.Range(0, max)
            from m in indexedMembers[i].TakeLast(1).DefaultIfEmpty()
            select CreateAccessors(m);

        IndexedPropertyDescriptor CreateAccessors(MemberInfo m) => m switch {
            PropertyInfo p =>
                new(Type: p.PropertyType,
                    Getter: obj => p.GetValue(obj),
                    Setter: (obj, value) => p.SetValue(obj, value)),
            FieldInfo f =>
                new(Type: f.FieldType,
                    Getter: obj => f.GetValue(obj),
                    Setter: (obj, value) => f.SetValue(obj, value)),
            _ => NoOperation,
        };
    }

    private record IndexedPropertyDescriptor(
        Type Type,
        Func<object, object?> Getter,
        Action<object, object?> Setter);
}

Usage remains the same as in Jeff's answer:

[JsonConverter(typeof(JsonArrayConverter<Person>))]
public class Person
{
    [JsonArrayIndex(0)]
    public string First { get; set; } = default!;
    
    [JsonArrayIndex(1)]
    public string Last { get; set; } = default!;
    
    [JsonArrayIndex(2)]
    public int Age { get; set; }
}

Carew

Your deleted question has popped up again stackoverflow.com/q/78827702/2836621 – Ferren

That Hoppe is not me. – Carew

Sorry, I wasn't suggesting it was you at all - I just thought you might want to "follow" it in case any eventual answer helps you out with your question. Good luck 🤞 – Ferren