Handling reads of Cosmos DB container with multiple types?

Question

I'd like to store several different object types in a single Cosmos DB container, as they are all logically grouped and make sense to read together by timestamp to avoid extra HTTP calls.

However, the Cosmos DB client API doesn't seem to provide an easy way of doing the reads with multiple types. The best solution I've found so far is to write your own CosmosSerializer and JsonConverter, but that feels clunky: https://thomaslevesque.com/2019/10/15/handling-type-hierarchies-in-cosmos-db-part-2/

Is there a more graceful way to read items of different types to a shared base class so I can cast them later, or do I have to take the hit?

Thanks!

Treadwell

Would the question be the same if you had a text file of JSON instead of a query to Cosmos? At the point of deserialization it seems you'd always need to choose the concrete type you want, otherwise a second deserialization operation would be needed. – Cymogene

More graceful? What does that mean to you. in objective terms? – Gomorrah

@Gomorrah more easily maintained, for one. Having my own serializer and converter means more maintenance work. If the reference implementation changes, I'd need to 1) know about it and 2) adopt the changes. – Treadwell

@NoahStahl Sure, but I'd expect that this, as a common pattern of data access in Cosmos, would have more sugar built around it. – Treadwell

I'd expect the reason is that there will be a performance hit if you try to treat items dynamically vs deserializing directly into an expected type. Personally, I use type as part of partition key and just design around strongly typed queries for the most part. – Cymogene

Unfortunately the hit you take there is on network requests if you need to query two separate containers for items that would otherwise be in one. – Treadwell

Answer 1

The way I do this is to create the ItemQueryIterator and FeedResponse objects as dynamic and initially read them untyped so I can inspect a "type" property that tells me what type of object to deserialize into.

In this example I have a single container that contains both my customer data as well as all their sales orders. The code looks like this.

        string sql = "SELECT * from c WHERE c.customerId = @customerId";

        FeedIterator<dynamic> resultSet = container.GetItemQueryIterator<dynamic>(
            new QueryDefinition(sql)
            .WithParameter("@customerId", customerId),
            requestOptions: new QueryRequestOptions 
            { 
                PartitionKey = new PartitionKey(customerId) 
            });

        CustomerV4 customer = new CustomerV4();
        List<SalesOrder> orders = new List<SalesOrder>();

        while (resultSet.HasMoreResults)
        {
            //dynamic response. Deserialize into POCO's based upon "type" property
            FeedResponse<dynamic> response = await resultSet.ReadNextAsync();
            foreach (var item in response)
            {
                if (item.type == "customer")
                {
                    customer = JsonConvert.DeserializeObject<CustomerV4>(item.ToString());
                    
                }
                else if (item.type == "salesOrder")
                {
                    orders.Add(JsonConvert.DeserializeObject<SalesOrder>(item.ToString()));
                }
            }
        }

Update:

You do not have to use dynamic types if want to create a "base document" class and then derive from that. Deserialize into the documentBase class, then check the type property check which class to deserialize the payload into.

You can also extend this pattern when you evolve your data models over time with a docVersion property.

Dekker

Any idea whether there are any perf implications of deserializing to dynamic first? – Treadwell

I don't know. I can't imagine it would be much though and this pattern is a typical one for handling multiple entities in a container. – Dekker

@MichaelTontchev dynamic used to be expensive but I believe I read somewhere that it is not the case any longer. Also, System.Text.Json is pretty fast and its what V3 is migrating to. – Amersfoort