JObject.ToBsonDocument dropping values

Question

I'm inserting raw JSON into a collection and finding that what is stored in the database is missing the values. For example, my collection is a collection of BsonDocuments:

_products = database.GetCollection<BsonDocument>("products");

The code to insert the JSON into the collection:

public int AddProductDetails(JObject json)
{
    var doc = json.ToBsonDocument(DictionarySerializationOptions.Document);
    _products.Insert(doc);
}

The JSON that is passed in looks like this:

{
  "Id": 1,
  "Tags": [
    "book",
    "database"
  ],
  "Name": "Book Name",
  "Price": 12.12
}

But, what is persisted in the collection is just the properties with no values.

{
  "_id": {
    "$oid": "5165c7e10fdb8c09f446d720"
  },
  "Id": [],
  "Tags": [
    [],
    []
  ],
  "Name": [],
  "Price": []
}

Why are the values being dropped?

Parliament

Answer 1

This does what I was expecting.

    public int AddProductDetails(JObject json)
    {
        BsonDocument doc = BsonDocument.Parse(json.ToString());
        _products.Insert(doc);
    }

Parliament

You should mark this as the answer (even though it's your own), as it seems to have solved my issue too. Thanks! – Foredoom

Great! This is what I was looking for. – Sebi

This is good but is there a more efficient version [3 years on]? copying to String might be an 'expensive' operation for large JSON? – Bilabial

@JamesWoodall - 5 years now :-) you can check out my answer – Underpinnings

@JamesWoodall you can use the Newtonsoft.Json.Bson nuget package to achieve this without serializing to/from a string. See my answer for an implementation of the mongo IBsonSerializer that uses this. – Esperanzaespial

Answer 2

I ran into this issue when I had a C# class with a property of type JObject.

My Solution was to create JObjectSerializer for MondoDB and add the attribute to the property so Mongo serializer uses it. I assume if I tried hard enough I could register the below serializer in Mongo as the global one for this type as well.

Register serializer for property processing:

[BsonSerializer(typeof(JObjectSerializer))]
public JObject AdditionalData { get; set; }

The serializer itself:

public class JObjectSerializer : SerializerBase<JObject> // IBsonSerializer<JObject>
{
    public override JObject Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
    {
        var myBSONDoc = BsonDocumentSerializer.Instance.Deserialize(context);
        return JObject.Parse(myBSONDoc.ToString());
    }

    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, JObject value)
    {
        var myBSONDoc = MongoDB.Bson.BsonDocument.Parse(value.ToString());
        BsonDocumentSerializer.Instance.Serialize(context, myBSONDoc);
    }
}

Entirety

BsonSerializer.RegisterSerializer(new JObjectSerializer()); is what you're looking for. Just stick it somewhere near your application startup and you won't need the property annotations. – Mime

Thank you! This did almost exactly what I needed. I had to tweak it slightly to handle null values correctly, but otherwise this is great! I added my updated version as an additional answer below. – Recurved

I..... searched..... alll..... day...... for this..... I read the docs, then I read them again. Then I read the github. Then I scoured the internet. Then I scoured stackoverflow. Then I did all of those things again 3 more times. sir... I think you sincerely. :) – Cataplexy

You can use the Newtonsoft.Json.Bson package to avoid serializing to/from a string. See my answer below for JObjectSerializer that uses this. – Esperanzaespial

Answer 3

The problem when using JObject.ToString, BsonDocument.Parse, etc. is the performance is not very good because you do the same operations multiple times, you do string allocations, parsing, etc.

So, I have written a function that converts a JObject to an IEnumerable<KeyValuePair<string, object>> (only using enumerations), which is a type usable by one of the BsonDocument constructors. Here is the code:

public static BsonDocument ToBsonDocument(this JObject jo)
{
    if (jo == null)
        return null;

    return new BsonDocument(ToEnumerableWithObjects(jo));
}

public static IEnumerable<KeyValuePair<string, object>> ToEnumerableWithObjects(this JObject jo)
{
    if (jo == null)
        return Enumerable.Empty<KeyValuePair<string, object>>();

    return new JObjectWrapper(jo);
}

private class JObjectWrapper : IEnumerable<KeyValuePair<string, object>>
{
    private JObject _jo;

    public JObjectWrapper(JObject jo)
    {
        _jo = jo;
    }

    public IEnumerator<KeyValuePair<string, object>> GetEnumerator() => new JObjectWrapperEnumerator(_jo);
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    public static object ToValue(JToken token)
    {
        object value;
        switch (token.Type)
        {
            case JTokenType.Object:
                value = new JObjectWrapper((JObject)token);
                break;

            case JTokenType.Array:
                value = new JArrayWrapper((JArray)token);
                break;

            default:
                if (token is JValue jv)
                {
                    value = ((JValue)token).Value;
                }
                else
                {
                    value = token.ToString();
                }
                break;
        }
        return value;
    }
}

private class JArrayWrapper : IEnumerable
{
    private JArray _ja;

    public JArrayWrapper(JArray ja)
    {
        _ja = ja;
    }

    public IEnumerator GetEnumerator() => new JArrayWrapperEnumerator(_ja);
}

private class JArrayWrapperEnumerator : IEnumerator
{
    private IEnumerator<JToken> _enum;

    public JArrayWrapperEnumerator(JArray ja)
    {
        _enum = ja.GetEnumerator();
    }

    public object Current => JObjectWrapper.ToValue(_enum.Current);
    public bool MoveNext() => _enum.MoveNext();
    public void Reset() => _enum.Reset();
}

private class JObjectWrapperEnumerator : IEnumerator<KeyValuePair<string, object>>
{
    private IEnumerator<KeyValuePair<string, JToken>> _enum;

    public JObjectWrapperEnumerator(JObject jo)
    {
        _enum = jo.GetEnumerator();
    }

    public KeyValuePair<string, object> Current => new KeyValuePair<string, object>(_enum.Current.Key, JObjectWrapper.ToValue(_enum.Current.Value));
    public bool MoveNext() => _enum.MoveNext();
    public void Dispose() => _enum.Dispose();
    public void Reset() => _enum.Reset();
    object IEnumerator.Current => Current;
}

Underpinnings

By far the most performant answer. Works like a charm. – Neurasthenic

Answer 4

Have you tried using the BsonSerializer?

using MongoDB.Bson.Serialization;
[...]
var document = BsonSerializer.Deserialize<BsonDocument>(json);

BsonSerializer works with strings, so if the JSON argument is a JObject(or JArray, JRaw etc) you have to serialize it with JsonConvert.SerializeObject()

Anesthetist

That doesn't compile. This does, but generates the same results.var doc = BsonSerializer.Deserialize<BsonDocument>(json.ToBsonDocument()); – Parliament

Argument 1: cannot convert from 'Newtonsoft.Json.Linq.JObject' to 'MongoDB.Bson.BsonDocument' – Parliament

Sorry, my fault, BsonSerializer works with json strings, so before deserializing it you have to serialize it with JsonConvert.SerializeObject(), then you should be okay. – Deiform

Answer 5

Here is an updated version of Andrew DeVries's answer that includes handling for serializing/deserializing null values.

public class JObjectSerializer : SerializerBase<JObject>
{
    public override JObject Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
    {
        if (context.Reader.CurrentBsonType != BsonType.Null)
        {
            var myBSONDoc = BsonDocumentSerializer.Instance.Deserialize(context);
            return JObject.Parse(myBSONDoc.ToStrictJson());
        }
        else
        {
            context.Reader.ReadNull();
            return null;
        }
    }

    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, JObject value)
    {
        if (value != null)
        {
            var myBSONDoc = BsonDocument.Parse(value.ToString());
            BsonDocumentSerializer.Instance.Serialize(context, myBSONDoc);
        }
        else
        {
            context.Writer.WriteNull();
        }
    }
}

The ToStrictJson() call is an extension method that wraps the call to the built-in BSON ToJson() method to include setting the output mode to strict. If this is not done, the parsing will fail because BSON type constructors will remain in the JSON output (ObjectId(), for example).

Here is the implementation of ToStrictJson() as well:

public static class MongoExtensionMethods
{
    /// <summary>
    /// Create a JsonWriterSettings object to use when serializing BSON docs to JSON.
    /// This will force the serializer to create valid ("strict") JSON.
    /// Without this, Object IDs and Dates are ouput as {"_id": ObjectId(ds8f7s9d87f89sd9f8d9f7sd9f9s8d)}
    ///  and {"date": ISODate("2020-04-14 14:30:00:000")} respectively, which is not valid JSON
    /// </summary>
    private static JsonWriterSettings jsonWriterSettings = new JsonWriterSettings()
    {
        OutputMode = JsonOutputMode.Strict
    };

    /// <summary>
    /// Custom extension method to convert MongoDB objects to JSON using the OutputMode = Strict setting.
    /// This ensure that the resulting string is valid JSON.
    /// </summary>
    /// <typeparam name="TNominalType">The type of object to convert to JSON</typeparam>
    /// <param name="obj">The object to conver to JSON</param>
    /// <returns>A strict JSON string representation of obj.</returns>
    public static string ToStrictJson<TNominalType>(this TNominalType obj)
    {
        return BsonExtensionMethods.ToJson<TNominalType>(obj, jsonWriterSettings);
    }
}

Recurved

Answer 6

I use the following. It's based on Simon's answer, thanks for the idea, and works in the same way, avoiding unnecessary serialization / deserialization into string.

It's just a bit more compact, thanks to Linq and C# 10:

public static BsonDocument ToBsonDocument(this JObject o) =>
    new(o.Properties().Select(p => new BsonElement(p.Name, p.Value.ToBsonValue())));

public static BsonValue ToBsonValue(this JToken t) =>
    t switch
    {
        JObject o => o.ToBsonDocument(),
        JArray a => new BsonArray(a.Select(ToBsonValue)),
        JValue v => BsonValue.Create(v.Value),
        _ => throw new NotSupportedException($"ToBsonValue: {t}")
    };

Duron

Answer 7

Most of the answers here involve serializing to and then deserializing from a string. Here is a solution that serializes to/from raw BSON instead. It requires the Newtonsoft.Json.Bson nuget package.

using System.IO;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Serializers;
using Newtonsoft.Json;
using Newtonsoft.Json.Bson;
using Newtonsoft.Json.Linq;

namespace Zonal.EventPublisher.Worker
{
    public class JObjectSerializer : SerializerBase<JObject>
    {
        public override JObject Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
        {
            using (var stream = new MongoDB.Bson.IO.ByteBufferStream(context.Reader.ReadRawBsonDocument()))
            using (JsonReader reader = new BsonDataReader(stream))
            {
                return JObject.Load(reader);
            }
        }

        public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, JObject value)
        {
            using (var stream = new MemoryStream())
            using (JsonWriter writer = new BsonDataWriter(stream))
            {
                value.WriteTo(writer);

                var buffer = new MongoDB.Bson.IO.ByteArrayBuffer(stream.ToArray());
                context.Writer.WriteRawBsonDocument(buffer);
            }
        }
    }
}

Don't forget to register the serializer with:

BsonSerializer.RegisterSerializer(new JObjectSerializer());

After that you can convert your JObject to a BsonDocument by using the MongoDB.Bson.BsonExtensionMethods.ToBsonDocument extension method:

var myBsonDocument = myJObject.ToBsonDocument()

And convert a BsonDocument back to a JObject by using the MongoDB.Bson.Serialization.BsonSerializer class:

var myJObject = BsonSerializer.Deserialize<JObject>(myBsonDocument);

Esperanzaespial