Using System.Text.Json to Serialize an IConfiguration back to Json

Question

I'm storing some IConfiguration as json in my sqlserver db so I can then bind them to some already constructed classes in order to provide dynamic settings.

At some point I might change the binded properties new at runtime and then update the db. The thing is that when i need to, the class might have more properties that aren't supposed to be bound and shouln't be serialized. I am therefore keeping the IConfiguration as a property of my class. Another reason why I'm using this approach is that I need to istantiate other children classes from the class that has loaded the configs, and save them to db when i do.

The thing is that when I serialize an IConfiguration i only get an empty json string like "{}". I suppose i could do some shenanigans leveraging .AsEnumerable() but isn't there a better way?

My sample code would look somewhat like this

public class ConfigurableClass
{

    public int ChildrenCount { get; set; } = 1069;
    public bool IsFast { get; set; } = false;
    public bool HasChildren { get; set; } = false;

    public int Id { get; }

    public ConfigurableClass(int id) { Id = id; }
}

static void Main(string[] args)
{

    IEnumerable<string> configs = SqlConfigLoader.LoadConfig();

    foreach (var str in configs)
    {
        Console.WriteLine("Parsing new Config:");

        var builder = new ConfigurationBuilder();

        IConfiguration cfg = builder.AddJsonStream(new MemoryStream(Encoding.Default.GetBytes(str)))
                .Build();

        var stepExample = new ConfigurableClass(9);

        cfg.Bind(stepExample);

        //do work with the class that might change the value of binded properties                   

        var updatedCfg = cfg;

        Console.WriteLine(JsonSerializer.Serialize(updatedCfg));

        Console.WriteLine();
    }

    Console.ReadLine();
}

Edit

I Also tried a diffent approach, by converting the IConfiguration to a nested dictionary like this

ublic static class IConfigurationExtensions
{
   public static Dictionary<string,object> ToNestedDicionary(this IConfiguration configuration)
   {
       var result = new Dictionary<string, object>();
       var children = configuration.GetChildren();
       if (children.Any()) 
           foreach (var child in children)
               result.Add(child.Key, child.ToNestedDicionary());
       else 
           if(configuration is IConfigurationSection section)
               result.Add(section.Key, section.Get(typeof(object)));

       return result;
   }        
}

But I lose the implicit type behind a given JsonElement:

if i serialize the resulting dictionary i get thing like "Property": "True" instead of "Property" : true

Glowworm

Answer 1

First, the why

Attempting to serialize the IConfiguration this way is not going to work how you want it to. Let's explore why.

Serializing Interfaces

Part of the reason you get no properties is because the generic type argument to Serialize is IConfiguration. In other words you are calling:

JsonSerializer.Serialize<IConfiguration>(updatedCfg)

When System.Text.Json serializes using a generic parameter it only (by default without any custom converters) serializes the public properties of that interface. In this case IConfiguration has no public properties (other than an indexer) so your output is empty json.

Using runtime-type information

Now, in general to get around this you would use the non-generic overload and pass the type. For example that would look like:

JsonSerializer.Serialize(updatedCfg, updatedCfg.GetType());

Or alternatively by using object as the type parameter:

JsonSerializer.Serialize<object>(updatedCfg);

System.Text.Json will then use the runtime type information in order to determine what properties to serialize.

The ConfigurationRoot

Now the second part of your problem is that this is unfortunately still not going to work due to how the configuration system is designed. The ConfigurationRoot class (the result of Build) can aggregate many configuration sources. The data is stored individually within (or even external to) each provider. When you request a value from the configuration it loops through each provider in order to locate a match.

All of this to say that the concrete/runtime type of your IConfiguration object will still not have the public properties you desire to serialize. In fact, passing the runtime type in this case will do worse than mimic the behavior of the interface as it will attempt to serialize the only public property of that type (ConfigurationRoot.Providers). This will give you a list of serialized providers, each typed as IConfigurationProvider and having zero public properties.

A potential solution

Since you are attempt to serialize the configuration that you are ultimately binding to an object, a workaround would be to re-serialize that object instead:

var stepExample = new ConfigurableClass(9);
cfg.Bind(stepExample);
var json1 = JsonSerializer.Serialize(stepExample, stepExample.GetType());
// or with the generic version which will work here
var json2 = JsonSerializer.Serialize(stepExample);

An alternative solution - AsEnumerable

IConfiguration is ultimately a collection of key value pairs. We can make use of the AsEnumerable extension method to create a List<KeyValuePair<string, string>> out of the entire configuration. This can later be deserialized and passed to something like AddInMemoryCollection

You'll need the Microsoft.Extensions.Configuration.Abstractions package (which is likely already transitively referenced) and the following using directive:

using Microsoft.Extensions.Configuration;

And then you can create a list of all the values (with keys in Section:Key format)

var configAsList = cfg.AsEnumerable().ToList();
var json = JsonSerializer.Serialize(configAsList);

Or you can use ToDictionary and serialize that instead.

var configAsDict = cfg.AsEnumerable().ToDictionary(c => c.Key, c => c.Value);
var json = JsonSerializer.Serialize(configAsDict);

Both formats will work with AddInMemoryCollection as that only requires an IEnumerable<KeyValuePair<string, string>> (which both types are). However, you will likely need the Dictionary format if you wish to use AddJsonFile/Stream as I don't think those support an array of key/value pairs.

Strings, strings and nothing but strings

You seem to be under the impression that IConfiguration objects are storing ints, bools, etc. (for example) corresponding to the JSON Element type. This is incorrect. All data within an IConfiguration is stored in stringified form. The base Configuration Provider classes all expect an IDictionary<string, string> filled with data. Even the JSON Configuration Providers perform an explicit ToString on the values.

The stringyly-typed values are turned into strongly-typed ones when you call Bind, Get<> or GetValue<>. These make use of the configuration binder which in turn uses registered TypeConverters and well know string parsing methods. But under the covers everything is still a string. This means it doesn't matter if your json file has a string property with value "True" or a boolean property with value true. The binder will appropriately convert the value when mapping to a boolean property.

Using the above dictionary serializing method will work as intended.

Grigson

the thing is that I wanted to leverage IConfig dynamism because the object is actually a derived class that might have some properties or not, I use the IConfiguration inside a Factory Class that determines the correct derived class and, after invoking the correct constructor, proceedes to bind the custom properties – Glowworm

IConfiguration seemed a good tool to do so, this way a custom derived class could have both its parsed settings and the raw IConfiguration. The objective is for every instance to be loaded correcly, have it mutate and be saved back to db – Glowworm

I guess our solutions have that common flaw (I updated my post), maybe I should try a different approach leveraging dynamic instead of a IConfiguration... – Glowworm

In response to your comment on your edit, IConfiguration has no concept of ints/bools. Everything is a string until it's bound to a container of the appropriate type. There's no reason that serializing to a dictionary and then back again wouldn't work because when it's rebound to your object the correct type converters will kick in. Whatever the json element's type is doesn't matter. It's always a string in the underlying configuration objects. It's calling Bind, Get<>, GetValue<> that have logic in them to perform conversions. true.ToString is "True" it will round-trip correctly – Grigson

And just to prove it, look at this line from the JSON file provider. This is part of a routine that returns a Dictionary<string, string> as required by the configuration infrastructure github.com/dotnet/runtime/blob/… you'll see it forcibly calls ToString on all the values you that you believe it preserves. It doesn't! The "dynamism" you refer to is part of the binder used to translate configs and not the configuration itself. My answer works :) – Grigson

Answer 2

just ran into a similar requirement. Serializing IConfiguration to send over a bus. Here's what I came up with

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration.Memory;

namespace A6k
{
    public class ConfigurationConverter : JsonConverter<IConfiguration>
    {
        public override IConfiguration Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var root = new ConfigurationRoot(new List<IConfigurationProvider>(new[] { new MemoryConfigurationProvider(new MemoryConfigurationSource()) }));

            var pathParts = new Stack<string>();
            string currentProperty = null;
            string currentPath = null;
            while (reader.Read() && (reader.TokenType != JsonTokenType.EndObject || pathParts.Count > 0))
            {
                switch (reader.TokenType)
                {
                    case JsonTokenType.PropertyName:
                        currentProperty = reader.GetString();
                        break;
                    case JsonTokenType.String:
                        if (pathParts.Count == 0)
                            root[currentProperty] = reader.GetString();
                        else
                            root[ConfigurationPath.Combine(currentPath, currentProperty)] = reader.GetString();
                        break;
                    case JsonTokenType.StartObject:
                        pathParts.Push(currentProperty);
                        currentPath = ConfigurationPath.Combine(pathParts);
                        break;
                    case JsonTokenType.EndObject:
                        pathParts.Pop();
                        currentPath = ConfigurationPath.Combine(pathParts);
                        break;
                }
            }
            return root;
        }

        public override void Write(Utf8JsonWriter writer, IConfiguration value, JsonSerializerOptions options)
        {
            if (value is IConfigurationSection section)
            {
                if (section.Value is null)
                    writer.WriteStartObject(section.Key);
                else
                {
                    writer.WriteString(section.Key, section.Value);
                    return;
                }
            }
            else
                writer.WriteStartObject();

            foreach (var child in value.GetChildren())
                Write(writer, child, options);

            writer.WriteEndObject();
        }
    }
}

It just uses a single memory config provider when deserializing. But the resulting IConfiguration behaves the same way as the sent instance.

Just add this as a converter to your serialization:

var json = JsonSerializer.Serialize(configuration, new JsonSerializerOptions
{
  Converters = { new ConfigurationConverter() },
  WriteIndented = true
});

or add as an attribute on a property

public class MyThing
{
    [JsonConverter(typeof(ConfigurationConverter))]
    public IConfiguration Config { get; set; }
}

Good luck!

Lyman

This is great! However, it runs into trouble if the initial IConfiguration is an IConfigurationSection (since IConifgurationSection is an IConfiguration). In such a case you end up with degenerate JSON that's missing opening and closing curly braces and includes the property name of the configuration section, which I'd argue it shouldn't (e.g. "sectionName": {"nested": "hi"}). I'll leave a fix as another solution and you can update as you see fit. – Baste

I haven't looked at this is years :smile: Yours certainly reads better. I'm sure I did something like this at the start. Can't remember why I ended up this way. It's just what worked for our use case. We never needed to deal with a subsection as the "root" of something to serialize, always a "whole" config. Glad it was useful to someone – Lyman

Answer 3

This answer builds on AndyPook's answer (https://mcmap.net/q/1399044/-using-system-text-json-to-serialize-an-iconfiguration-back-to-json), but the Write method has been updated to handle the case where the root IConfiguration given IS an IConfigurationSection. I think it's generally easier to reason about as well.

Without this change, serializing an IConfigurationSection as an IConfiguration results in degenerate JSON that's missing opening and closing braces and includes a property name for the section (which I don't think it should). For example, trying to serialize the sectionName section of a config like {"sectionName": {"nested": "hi"}} results in "sectionName": {"nested": "hi"} and I think it should instead result in {"nested": "hi"}.

Only the Write method is different.

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration.Memory;

namespace A6k
{
    public class ConfigurationConverter : JsonConverter<IConfiguration>
    {
        public override IConfiguration Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var root = new ConfigurationRoot(new List<IConfigurationProvider>(new[] { new MemoryConfigurationProvider(new MemoryConfigurationSource()) }));

            var pathParts = new Stack<string>();
            string currentProperty = null;
            string currentPath = null;
            while (reader.Read() && (reader.TokenType != JsonTokenType.EndObject || pathParts.Count > 0))
            {
                switch (reader.TokenType)
                {
                    case JsonTokenType.PropertyName:
                        currentProperty = reader.GetString();
                        break;
                    case JsonTokenType.String:
                        if (pathParts.Count == 0)
                            root[currentProperty] = reader.GetString();
                        else
                            root[ConfigurationPath.Combine(currentPath, currentProperty)] = reader.GetString();
                        break;
                    case JsonTokenType.StartObject:
                        pathParts.Push(currentProperty);
                        currentPath = ConfigurationPath.Combine(pathParts);
                        break;
                    case JsonTokenType.EndObject:
                        pathParts.Pop();
                        currentPath = ConfigurationPath.Combine(pathParts);
                        break;
                }
            }
            return root;
        }

        public override void Write(Utf8JsonWriter writer, IConfiguration value, JsonSerializerOptions options)
        {
            writer.WriteStartObject();
            foreach (var child in value.GetChildren())
            {    
                writer.WritePropertyName(child.Key);
                if (child.Value is null)
                {
                    Write(writer, child, options);
                }
                else
                {
                    writer.WriteStringValue(child.Value);
                }
            }
            writer.WriteEndObject();
        }
    }
}

Baste

Answer 4

This answer is based on tdg5's answer. I have a task to use some of configuration's section's as a response templates for my rest service. So i have to serialize an IConfigurationSection back to json and i need to get exactly the same json as stored in configuration with support for all json types. But there is two small problems:

1. Section store empty json object and empty json array in the same way, so i have to put "$empty$" placeholder in my empty array that will be removed in serialization process.
2. Section store empty string and null value as empty strings. Yet again i have to put "$empty$" placeholder in empty strings to serialize them appropriately.

Here is my code:

private JsonNode GetSectionAsJson(IConfigurationSection section)
{
    using Stream stream = new MemoryStream();
    using Utf8JsonWriter writer = new(stream);
    Write(writer, section);
    writer.Flush();
    stream.Seek(0, SeekOrigin.Begin);
    return JsonNode.Parse(stream)!;
}

private void Write(Utf8JsonWriter writer, IConfigurationSection section)
{
    bool isArray = section.GetChildren().FirstOrDefault()?.Key == "0";
    if (isArray)
    {
        WriteArray(writer, section);
    }
    else
    {
        bool isObj = section.Value == null;
        if (isObj)
        {
            WriteObject(writer, section);
        }
        else
        {
            WriteValue(writer, section);
        }
    }
}

private void WriteObject(Utf8JsonWriter writer, IConfigurationSection section)
{
    writer.WriteStartObject();
    foreach (var child in section.GetChildren())
    {
        writer.WritePropertyName(child.Key);
        Write(writer, child);
    }
    writer.WriteEndObject();
}

private void WriteArray(Utf8JsonWriter writer, IConfigurationSection section)
{
    writer.WriteStartArray();
    bool isEmptyArray = section.GetChildren().FirstOrDefault()?.Value == "$empty$";
    if (isEmptyArray)
    {
        writer.WriteEndArray();
        return;
    }

    foreach (var child in section.GetChildren())
    {
        Write(writer, child);
    }
    writer.WriteEndArray();
}

private void WriteValue(Utf8JsonWriter writer, IConfigurationSection section)
{
    if (string.IsNullOrEmpty(section.Value))
    {
        writer.WriteNullValue();
        return;
    }
    if (bool.TryParse(section.Value, out bool boolVal))
    {
        writer.WriteBooleanValue(boolVal);
        return;
    }
    if (decimal.TryParse(section.Value, _parseDecimalCultureInfo, out decimal decVal))
    {
        writer.WriteNumberValue(decVal);
        return;
    }
    writer.WriteStringValue(section.Value != "$empty$" ? section.Value : "");
}

Canalize

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Nightclub