Get .NET Core JsonSerializer to serialize private members

Question

I have a class with a private List<T> property which I would like to serialize/deserialize using the JsonSerializer. Use of the JsonPropertyAttribute doesn't seem to be supported in .NET Core. So how can I have my private list property serialized?

I'm using System.Text.Json for this.

Guardafui

Are you using Newtonsoft.Json or System.Text.Json? – Hepza

Serialised but not deserialised? Add a read-only public property that returns your private List on get. – Roesch

@HereticMonkey I'm using System.Text.Json. Added this to the question. – Guardafui

System.Text.Json doesn't support internal and private getters and setters out of the box – Javed

FYI: github.com/dotnet/runtime/pull/34675 – Swirl

Answer 1

It seems System.Text.Json does not support private property serialization.

https://learn.microsoft.com/tr-tr/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to#internal-and-private-property-setters-and-getters

But as the Microsoft's document says, you can do it with custom converters.

https://www.thinktecture.com/en/asp-net/aspnet-core-3-0-custom-jsonconverter-for-the-new-system_text_json/

Code snippet for serialization & deserialization;

  public class Category
    {
        public Category(List<string> names)
        {
            this.Names1 = names;
        }

        private List<string> Names1 { get; set; }
        public string Name2 { get; set; }
        public string Name3 { get; set; }
    }


 public class CategoryJsonConverter : JsonConverter<Category>
    {
        public override Category Read(ref Utf8JsonReader reader,
                                      Type typeToConvert,
                                      JsonSerializerOptions options)
        {
                       var name = reader.GetString();

            var source = JsonSerializer.Deserialize<Dictionary<string, JsonElement>>(name);

            var category = new Category(null);

            var categoryType = category.GetType();
            var categoryProps = categoryType.GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);

            foreach (var s in source.Keys)
            {
                var categoryProp = categoryProps.FirstOrDefault(x => x.Name == s);

                if (categoryProp != null)
                {
                    var value = JsonSerializer.Deserialize(source[s].GetRawText(), categoryProp.PropertyType);

                    categoryType.InvokeMember(categoryProp.Name,
                        BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.SetProperty | BindingFlags.Instance,
                        null,
                        category,
                        new object[] { value });
                }
            }

            return category;
        }

        public override void Write(Utf8JsonWriter writer,
                                   Category value,
                                   JsonSerializerOptions options)
        {
            var props = value.GetType()
                             .GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)
                             .ToDictionary(x => x.Name, x => x.GetValue(value));

            var ser = JsonSerializer.Serialize(props);

            writer.WriteStringValue(ser);
        }
    }

static void Main(string[] args)
    {
        Category category = new Category(new List<string>() { "1" });
        category.Name2 = "2";
        category.Name3 = "3";

        var opt = new JsonSerializerOptions
        {
            Converters = { new CategoryJsonConverter() },
            Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
        };

        var json = JsonSerializer.Serialize(category, opt);

        var obj = JsonSerializer.Deserialize<Category>(json, opt);

        Console.WriteLine(json);
        Console.ReadKey();
    }

Result;

"{\"Names1\":[\"1\"],\"Name2\":\"2\",\"Name3\":\"3\"}"

Fearfully

I've read the article and I'm not sure whether this is going to help me to serialize/deserialize a private property. To me it seems that a custom JsonConverter allows me to override how a certain property is serialized but not if (in case of a private property this is not the case). Please do correct me if I'm wrong. – Guardafui

@Guardafui - you don't write a converter for the property, you write the converter for the entire object and serialize it manually (including all private fields) via the converter. – Calibre

@Guardafui i think the article may be helps you i didn't dig it deeply. And update my answer with some code snippet for serialization, may be it guides you to go further because there is no way (System.Text Json does not supply that) except your way :) – Fearfully

@Fearfully Oh you are right! I got that working. To make it a useful and complete answer for the next guy with the same problem, I'd like to ask you to provide some deserialization example. And do you know why the JSON output has all quotes escaped? Instead of {\"Names1\": ...} I think it should be{"Names1":...}. I tried fiddling around with the JavascriptEncoder but to no avail. – Guardafui

@Guardafui ok, I updated my answer with code snippet for deserialization. Please check if it is ok for your question. – Fearfully

Answer 2

In .NET 7 and later Microsoft has added the ability to programmatically customize the serialization contract that System.Text.Json creates for each .NET type. Using this API you can add a typeInfo modifier to serialize selected (or all) private properties of selected types.

E.g., you might want to:

1. Serialize all private properties marked with some custom attribute.

2. Serialize all private properties of a specific type.

3. Serialize a specific private property of a specific type by name.

Given these requirements, define the following attribute and modifiers:

[System.AttributeUsage(System.AttributeTargets.Property, AllowMultiple = false, Inherited = true)]
public class JsonIncludePrivatePropertyAttribute : System.Attribute { }

public static partial class JsonExtensions
{
    public static Action<JsonTypeInfo> AddPrivateProperties<TAttribute>() where TAttribute : System.Attribute => typeInfo => 
    {
        if (typeInfo.Kind != JsonTypeInfoKind.Object)
            return;
        foreach (var type in typeInfo.Type.BaseTypesAndSelf().TakeWhile(b => b != typeof(object)))
            AddPrivateProperties(typeInfo, type, p => Attribute.IsDefined(p, typeof(TAttribute)));
    };

    public static Action<JsonTypeInfo> AddPrivateProperties(Type declaredType) => typeInfo => 
        AddPrivateProperties(typeInfo, declaredType, p => true);
    
    public static Action<JsonTypeInfo> AddPrivateProperty(Type declaredType, string propertyName) => typeInfo => 
    {
        if (typeInfo.Kind != JsonTypeInfoKind.Object || !declaredType.IsAssignableFrom(typeInfo.Type))
            return;
        var propertyInfo = declaredType.GetProperty(propertyName, BindingFlags.Instance | BindingFlags.NonPublic);
        if (propertyInfo == null)
            throw new ArgumentException(string.Format("Private roperty {0} not found in type {1}", propertyName, declaredType));
        if (typeInfo.Properties.Any(p => p.GetMemberInfo() == propertyInfo))
            return;
        AddProperty(typeInfo, propertyInfo);
    };

    static void AddPrivateProperties(JsonTypeInfo typeInfo, Type declaredType, Func<PropertyInfo, bool> filter)
    {
        if (typeInfo.Kind != JsonTypeInfoKind.Object || !declaredType.IsAssignableFrom(typeInfo.Type))
            return;
        var propertyInfos = declaredType.GetProperties(BindingFlags.Instance | BindingFlags.NonPublic);
        foreach (var propertyInfo in propertyInfos.Where(p => p.GetIndexParameters().Length == 0 && filter(p)))
            AddProperty(typeInfo, propertyInfo);
    }
    
    static void AddProperty(JsonTypeInfo typeInfo, PropertyInfo propertyInfo)
    {
        if (propertyInfo.GetIndexParameters().Length > 0)
            throw new ArgumentException("Indexed properties are not supported.");
        var ignore = propertyInfo.GetCustomAttribute<JsonIgnoreAttribute>();
        if (ignore?.Condition == JsonIgnoreCondition.Always)
            return;
        var name = propertyInfo.GetCustomAttribute<JsonPropertyNameAttribute>()?.Name 
            ?? typeInfo.Options?.PropertyNamingPolicy?.ConvertName(propertyInfo.Name) 
            ?? propertyInfo.Name;
        var property = typeInfo.CreateJsonPropertyInfo(propertyInfo.PropertyType, name);
        property.Get = CreateGetter(typeInfo.Type, propertyInfo.GetGetMethod(true));
        property.Set = CreateSetter(typeInfo.Type, propertyInfo.GetSetMethod(true));
        property.AttributeProvider = propertyInfo;
        property.CustomConverter = propertyInfo.GetCustomAttribute<JsonConverterAttribute>()?.ConverterType is {} converterType
            ? (JsonConverter?)Activator.CreateInstance(converterType)
            : null;
        // TODO: handle ignore?.Condition == JsonIgnoreCondition.Never,  WhenWritingDefault, or WhenWritingNull by setting property.ShouldSerialize appropriately
        // TODO: handle JsonRequiredAttribute, JsonNumberHandlingAttribute
        typeInfo.Properties.Add(property);
    }

    delegate TValue RefFunc<TObject, TValue>(ref TObject arg);
    
    static Func<object, object?>? CreateGetter(Type type, MethodInfo? method)
    {
        if (method == null)
            return null;
        var myMethod = typeof(JsonExtensions).GetMethod(nameof(JsonExtensions.CreateGetterGeneric), BindingFlags.NonPublic | BindingFlags.Static)!;
        return (Func<object, object?>)(myMethod.MakeGenericMethod(new[] { type, method.ReturnType }).Invoke(null, new[] { method })!);
    }

    static Func<object, object?> CreateGetterGeneric<TObject, TValue>(MethodInfo method)
    {
        if (method == null)
            throw new ArgumentNullException();
        if(typeof(TObject).IsValueType)
        {
            // https://mcmap.net/q/753535/-how-can-i-create-an-open-delegate-from-a-struct-39-s-instance-method
            // https://mcmap.net/q/753536/-quot-uncurrying-quot-an-instance-method-in-net/1212396#1212396
            var func = (RefFunc<TObject, TValue>)Delegate.CreateDelegate(typeof(RefFunc<TObject, TValue>), null, method);
            return (o) => {var tObj = (TObject)o; return func(ref tObj); };
        }
        else
        {
            var func = (Func<TObject, TValue>)Delegate.CreateDelegate(typeof(Func<TObject, TValue>), method);
            return (o) => func((TObject)o);
        }
    }

    static Action<object,object?>? CreateSetter(Type type, MethodInfo? method)
    {
        if (method == null)
            return null;
        var myMethod = typeof(JsonExtensions).GetMethod(nameof(JsonExtensions.CreateSetterGeneric), BindingFlags.NonPublic | BindingFlags.Static)!;
        return (Action<object,object?>)(myMethod.MakeGenericMethod(new [] { type, method.GetParameters().Single().ParameterType }).Invoke(null, new[] { method })!);
    }
    
    static Action<object,object?>? CreateSetterGeneric<TObject, TValue>(MethodInfo method)
    {
        if (method == null)
            throw new ArgumentNullException();
        if (typeof(TObject).IsValueType)
        {
            // TODO: find a performant way to do this.  Possibilities:
            // Box<T> from Microsoft.Toolkit.HighPerformance
            // https://mcmap.net/q/510889/-how-to-mutate-a-boxed-struct-using-il
            return (o, v) => method.Invoke(o, new [] { v });
        }
        else
        {
            var func = (Action<TObject, TValue?>)Delegate.CreateDelegate(typeof(Action<TObject, TValue?>), method);
            return (o, v) => func((TObject)o, (TValue?)v);
        }
    }

    static MemberInfo? GetMemberInfo(this JsonPropertyInfo property) => (property.AttributeProvider as MemberInfo);
    
    static IEnumerable<Type> BaseTypesAndSelf(this Type? type)
    {
        while (type != null)
        {
            yield return type;
            type = type.BaseType;
        }
    }
}

Then, if your model looks like e.g.:

public partial class Model
{
    List<int> PrivateList { get; set; } = new();

    [JsonIgnore]  // For testing purposes only
    public List<int> SurrogateList { get => PrivateList; set => PrivateList = value; }
}

Then you could mark PrivateList with [JsonIncludePrivateProperty]:

public partial class Model
{
    [JsonIncludePrivateProperty]
    List<int> PrivateList { get; set; } = new();

And serialize using the following options:

var options = new JsonSerializerOptions
{
    TypeInfoResolver = new DefaultJsonTypeInfoResolver
    {
        Modifiers = { JsonExtensions.AddPrivateProperties<JsonIncludePrivatePropertyAttribute>() },
    },
};

Or if you can't change your model, you could include all its private properties as follows:

var options = new JsonSerializerOptions
{
    TypeInfoResolver = new DefaultJsonTypeInfoResolver
    {
        Modifiers = { JsonExtensions.AddPrivateProperties(typeof(Model)) },
    },
};

Or just the property named PrivateList as follows:

var options = new JsonSerializerOptions
{
    TypeInfoResolver = new DefaultJsonTypeInfoResolver
    {
        Modifiers = { JsonExtensions.AddPrivateProperty(typeof(Model), "PrivateList") },
    },
};

With any of the above options, the JSON generated will be e.g. {"PrivateList":[1,2,3]}.

Notes:

• Automatically serializing all private properties of all types is not recommended, but if you need to do it for some reason, use the following modifier:

  public static Action<JsonTypeInfo> AddPrivateProperties() => typeInfo => 
  {
      if (typeInfo.Kind != JsonTypeInfoKind.Object)
          return;
      foreach (var type in typeInfo.Type.BaseTypesAndSelf().TakeWhile(b => b != typeof(object)))
          AddPrivateProperties(typeInfo, type, p => true);
  };
  

• As of .NET 7 there is no access to System.Text.Json's constructor metadata, so there does not seem to be a way to serialize a private property and have it deserialized as a constructor parameter.

• For a typeInfo modifer that causes private fields to be serialized, see the documentation example Customize a JSON contract: Example: Serialize private fields.

• It is possible to have private properties with the same name in base and derived types. If you try to serialize the private properties of both you may get an exception

  System.InvalidOperationException: The JSON property name for 'Type.PropertyName' collides with another property.

  If this happens, you will map one of the properties to a different name, e.g. by adding [JsonPropertyName("SomeAlternateName")] to one of them.

Demo fiddle here.

Calibre

Answer 3

System.Text.Json partially supports private property serialization starting with .NET 5, according to Microsoft documentation:

System.Text.Json supports private and internal property setters and getters via the [JsonInclude] attribute.

Note the very specific wording of the above documentation. It means that if you have the following property:

private string MyProperty { get; set; }

then [JsonInclude] will not work. However if you have this property declared as follows:

public string MyProperty { private get; private set; }

then it works as expected.

Find more details here.

Replacement

Doesn't work for me: 'The non-public property 'TenantsSource' on type '...SubscriptionModel' is annotated with 'JsonIncludeAttribute' which is invalid.'. Looks like [JsonInclude] works if either the getter or setter are private, but not if both are private. – Silverts

To be honest I'm not a fan of using such attributes on domain models anyway. For such private model serializations we are still using Newtonsoft json. – Replacement

[JsonInclude] - "non-public getters and setters can be used ... Non-public properties are not supported." - doc – Otherwise

@IanKemp - In C# you can't specify modifiers for both setters and getters, if you do you will get a compilation error Cannot specify accessibility modifiers for both accessors of the property or indexer 'Model.MyProperty', see dotnetfiddle.net/hZEMFH. – Calibre

Answer 4

Although you cannot serialize a private field directly as is, you can do it indirectly.

You need to provide a public property for the field and a constructor as in the following example:

class MyNumbers
{
    // This private field will not be serialized
    private List<int> _numbers;

    // This public property will be serialized
    public IEnumerable<int> Numbers => _numbers;

    // The serialized property will be recovered with this dedicated constructor
    // upon deserialization. Type and name must be the same as the public property.
    public MyNumbers(IEnumerable<int> Numbers = null)
    {
        _numbers = Numbers as List<int> ?? Numbers?.ToList() ?? new();
    }
}

The following code demonstrates how that works:

string json;
// Serialization
{
    MyNumbers myNumbers = new(new List<int> { 10, 20, 30});
    json = JsonSerializer.Serialize(myNumbers);
    Console.WriteLine(json);
}
// Deserialization
{
    var myNumbers2 = JsonSerializer.Deserialize<MyNumbers>(json);
    foreach (var number in myNumbers2.Numbers)
        Console.Write(number + "  ");
}

Output:

{"Numbers":[10,20,30]}
10  20  30

If you want to detract people from accessing your private data, you can change the name to something explicitly forbidden like __private_numbers.

class MyNumbers2
{
    private List<int> _numbers;

    public IEnumerable<int> __private_numbers => _numbers;

    public MyNumbers2(IEnumerable<int> __private_numbers = null)
    {
        _numbers = __private_numbers as List<int> ?? __private_numbers?.ToList() ?? new();
    }
}

If an external coder is fool enough to access that private data as if it was part of the normal programming interface of that class, then shame on him. You are in your plain right to change that "private interface" without any guilt. And he can't mess with your internal list either, with an IEnumerable.

In most situations, that should be enough.

Dias