Slice array in mongodb after $addToSet update

Question

I have a document with a structure similar to this

{
    "brand": "BMW",
    "models": ["320","545"]
}

Models must be unique, and I am using the following query when adding new items,

db.cars.update(
    {brand:'BMW'},
    {
        $addToSet: {
            models: '750'
        }
    },
    {upsert:true}
);

That would give

{
    "brand": "BMW",
    "models": ["320","545","750"]
}

Question:

How can I limit the total number of items 'models' can have? Say I want to keep only the last 3 added models. So if I insert a new model '135' I would end up with

{
    "brand": "BMW",
    "models": ["545","750","135"]
}

I read about the $slice modifier however it appears to be only available when using $push and not $addToSet

Eleanor

Answer 1

Interesting question as this is actually the subject of a bug that has been raised an an actual request for addition. As for the request, I wouldn't hold your breath on getting this behavior added.

The bug is in that you could actually issue the following:

db.cars.update(
    {brand:'BMW'},
    { $addToSet: { models: { $each: ['200'], $slice: 3 } } },
    {upsert: true}
)

and addToSet would simply work whilst ignoring the modifier, or warning. But of course you would now have four elements in your array.

The comments, from the CTO no less, point to a conflict in terms.

Thus the only thing I can think of is to first test for the element in the array using $elemMatch in a find and if both the document exists and the $elemMatch find is not true then update with a regular $push and slice. Otherwise insert the new document or leave it alone

db.cars.update(
   {brand:'BMW'},
   {$push: { models: { $each: ['750'], $slice: -3 } }}
)

That would result in two more ops over the wire. Alternately pull down the document and manipulate the array yourself which would be one more op. Either way you loose the upsert magic and have to do things manually.

Naker

thank you for your answer, I have been using mongo only for a couple of days and thought something like this would be trivial. From the links I gather that there seems to be a particular distinction between arrays and sets, which I did not know about. – Eleanor

The SERVER-8512: support $slice/sort in $addToSet "request" you mention was already closed as "won't fix" so definitely don't hold your breath for that ;-). Sets are currently implemented as arrays, but it doesn't make sense to mix $addToSet modifier with array $slice mechanics. Sets should be considered unordered. – Luckin

Answer 2

Had the same problem. Here is the workaround that I did (works more efficient with bulk operations obviously, but can work on single operations also):

Basically you add dummy empty $push operator and use $slice on it. But since $addToSet is NOT guarantee order, we also need to add $sort on it.

As far as I understand, you need the model names in the order they inserted. So you will need to add a timestamp to it (we can use epoch). So we convert the model to an object: { name: '750', ts: 1552428379 }

bulk = [
  {
    updateOne: {
      filter: { brand: 'BMW' },
      update: {
        $addToSet: {
          models: { name: '750', ts: 1552428379 }
        }
      },
      upsert: true
    }
  },
  {
    updateOne: {
      filter: { brand: 'BMW' },
      update: {
        $push: {
          models: {
            $each: [],
            $sort: { ts: 1 },
            $slice: -3
          }
        }
      }
    }
  },
]
db. cars.bulkWrite(bulk)

FYI, it looks like you can't use $addToSet and $push in the same operation. That is why we use 2 updates.

IMPORTANT: make sure you are not using 'ordered: false' in the bulk operation!

Vaccine

Answer 3

I use the following strategy. I will use _id for filtering instead of brand to make use of the unique index (or you can create your own unique index). An example document in db.cars:

{
    "_id": "BMW",
    "models": ["320","545"]
}

The update operation to set-like push and slice:

// js-like pseudocode below

let added;
try {
    const updateResult = db.cars.updateOne(
        { 
            "_id": "BMW",
            "models": { $ne: 750 }
        },
        {
            $push: {
                "models": {
                    $each: [750],
                    $slice: -3 // slice last 3 elements of the array
                }
            }
        },
        {
            upsert: true
        }
    );
    added = updateResult.matchedCount > 0 || updateResult.upsertedId != null
} catch(e) {
    if (e.code === 11000) {
        added = false;
    } else {
        throw e
    }
}
print(added)

It's very important to have "models": { $ne: 750 } filter and upsert: true update option, so we can use E11000 duplicate key error as a clear signal that we're trying to add a duplicate entry to the set array.

Yoke