The JSON value could not be converted to System.Byte[]

Question

I'm getting the following exception when trying to deserialize the following JSON when dealing with byte arrays, what's wrong?

public class Program
{
    public static void Main()
    {
        var root = JsonSerializer.Deserialize<JsonRoot>(@"{ ""ByteArray"": [1] } ");
    }

    public class JsonRoot
    {
        public byte[] ByteArray {get;set;}  
    }
}

Unhandled exception. System.Text.Json.JsonException: The JSON value could not be converted to System.Byte[]. Path: $.ByteArray | LineNumber: 0 | BytePositionInLine: 16.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'StartArray' as a string.
   at System.Text.Json.Utf8JsonReader.TryGetBytesFromBase64(Byte[]& value)
   at System.Text.Json.Utf8JsonReader.GetBytesFromBase64()
   at System.Text.Json.Serialization.Converters.JsonConverterByteArray.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.JsonPropertyInfoNotNullable`4.OnRead(ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.JsonPropertyInfo.Read(JsonTokenType tokenType, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& readStack, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
   at System.Text.Json.JsonSerializer.ReadCore(Type returnType, JsonSerializerOptions options, Utf8JsonReader& reader)
   at System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Program.Main()
Command terminated by signal 6

Balling

Weird, it works with int[]. This is probably a limitation or bug in the serializer. Have you searched the github repo? – Vitriform

I've created the sample JsonRoot object var test = new JsonRoot { ByteArray = new[] { (byte)1 } }; and serialize it for the test, and got {"ByteArray":"AQ=="} result. It seems, that under the hood System.Text.Json uses special rules for byte arrays. Maybe because it's using Utf8JsonReader/Utf8JsonWriter – Petrosal

I was right with my first comment. I've added an answer with a link. When I get time, I'll add a custom converter which does what you want. – Vitriform

I've searched around but can't find any issue related to this. – Balling

@KevinSmith you didn't search well enough ;) Again, I've added an answer with that link. I'll add a converter later. – Vitriform

@KevinSmith Added the converter. You might want to extend it a bit for better null and error handling but this should get you started. – Vitriform

Answer 1

With Sytem.Text.Json a byte array (byte[]) will be serialized as base64 string. They stated that they won't add support for byte[] to be serialized as number array in a github issue.

Here's a custom converter that should get you started. Maybe you could optimize the reading a bit but the performance hit taken by this approach shouldn't be too bad. You might want to add null and error handling but you get the idea.

To apply the custom converter, you'll have to add them to the JsonSerializerOptions. Please refer to this docs page for that.

public class ByteArrayConverter : JsonConverter<byte[]>
{
    public override byte[] Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        short[] sByteArray = JsonSerializer.Deserialize<short[]>(ref reader);
        byte[] value = new byte[sByteArray.Length];
        for (int i = 0; i < sByteArray.Length; i++)
        {
            value[i] = (byte)sByteArray[i];
        }

        return value;
    }

    public override void Write(Utf8JsonWriter writer, byte[] value, JsonSerializerOptions options)
    {
        writer.WriteStartArray();

        foreach (var val in value)
        {
            writer.WriteNumberValue(val);
        }

        writer.WriteEndArray();
    }
}

Vitriform

Thanks, awesome answer - didn't know it only did base64! – Balling

@KevinSmith this isn't "only", it's the standard: When it is required that an I-JSON protocol element contain arbitrary binary data, it is RECOMMENDED that this data be encoded in a string value in base64url Anything else is a custom encoding that no other application would understand. – Baculiform

RFC 7493 link for Base64 recommendation: rfc-editor.org/rfc/rfc7493#section-4.4 – Typography

Answer 2

Using ASP.NET 6:

In your model:

public class JsonRoot
{
    // Add this attribute to your byte array(s).
    [JsonConverter(typeof(JsonToByteArrayConverter))]
    public byte[] ByteArray {get;set;}  
}

Add a new class:

using System.Text.Json;
using System.Text.Json.Serialization;

internal sealed class JsonToByteArrayConverter : JsonConverter<byte[]?>
{
    // Converts base64 encoded string to byte[].
    public override byte[]? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (!reader.TryGetBytesFromBase64(out byte[]? result) || result == default)
        {
            throw new Exception("Add your fancy exception message here...");
        }
        return result;
    }

    // Converts byte[] to base64 encoded string.
    public override void Write(Utf8JsonWriter writer, byte[]? value, JsonSerializerOptions options)
    {
        writer.WriteBase64StringValue(value);
    }
}

Immaterial

System.InvalidOperationException: 'Cannot get the value of a token type 'StartArray' as a string.' on the line if (!reader.TryGetBytesFromBase64(out byte[]? result) || result == default) – Ceaseless

Answer 3

a simple solution:

Use Newtonsoft.Json.JsonConvert.DeserializeObject<...> instead of JsonSerializer.Deserialize<...>. it will solve the problem.

Territerrible

Even though this will work, there are other problems that arise with using Newtonsoft such as excessive memory allocations etc... – Balling