Batch insert/update using Mongoid?

Question

I googled and all others, but I didn't find the answer. The question is:

Hi, how can I do batch insert with Mongoid to MongoDB?

Feria

Answer 1

You can insert a batch array of hashes using the ruby mongo driver's insert method. From any Mongoid class, you can call collection to access it.

batch = [{:name => "mongodb"}, {:name => "mongoid"}]  
Article.collection.insert(batch)

Lectureship

A very important point to remember is that by doing this you are bypassing mongoid.yml options. So if you use it, make sure to inform them if you need, e.g.: Article.with(safe: true).collection.insert(batch) – Strain

I'm receiving NoMethodError: undefined method `insert' – Sestos

Me too. Asked a question about it #33894632 – Mokas

For undefined method 'insert' try bellow answer using create – Lacework

@HarisKrajina - In the updated version of mongoid you will need to use the 'insert_many' method. Hope this helps, – Cohby

@maxkaplan - Mongoid does not seem to have this anymore: docs.mongodb.com/ruby-driver/master/tutorials/6.1.0/… – Pitre

As of Mongoid 7 insert has been replace with insert_many: Article.collection.insert_many(batch) – Pitre

@HarisKrajina - create does not do batch insert. – Pitre

Answer 2

If you want to batch insert Mongoid documents (models) instead of hashes, call your model's as_document method before placing it into array:

@page_views << page_view.as_document

...

PageView.collection.insert(@page_views)

Tallyho

I get this error undefined method `as_document' for #<Array:0x10a40f870> any solution ? – Litigation

@AbhayKumar as_document doesn't work against an array, only single objects. If you already have an array of objects, you can call it like so: PageView.collection.insert(@page_views.map(&:as_document)) which will map all of the objects to documents prior to insertion. – Barret

@Barret Does mongoid or mongodb solve out the fact that an object already exists and perform an update? Or is it necessary to filter those out? – Keefer

@Keefer As far as I'm aware Mongoid ignores all validations in this instance (including uniqueness checks), so I doubt it. I can't test as I'm not using either in my current project, but I would say insertion of duplicates would be the most likely outcome by far :) Try it out and let me know how it goes! – Barret

RE: last two commenters, collection.insert will create duplicates and performs no update or find_or_create_by like operations. But it is a lot faster! I ran a benchmark test and found that I could insert 1000 records with collection.insert in 0.07 seconds, whereas inserting 1000 records with find_or_create_by took 3.35 seconds. – Hatchment

Answer 3

You can use this:

books = [{:name => "Harry Potter"}, {:name => "Night"}]  
Book.collection.insert_many(books)

And I find that "insert" does not work for me(Monogoid 5.1.3):

NoMethodError: undefined method `insert' for # <Mongo::Collection:0x007fbdbc9b1cd0>
Did you mean?  insert_one
               insert_many
               inspect

This is the source code from "lib/mongo/collection.rb":

# Insert the provided documents into the collection.
#
# @example Insert documents into the collection.
#   collection.insert_many([{ name: 'test' }])
#
# @param [ Array<Hash> ] documents The documents to insert.
# @param [ Hash ] options The insert options.
#
# @return [ Result ] The database response wrapper.
#
# @since 2.0.0
def insert_many(documents, options = {})
  inserts = documents.map{ |doc| { :insert_one => doc }}
  bulk_write(inserts, options)
end

Geostatics

When we use document.collection.insert_many it skips after_save callback. How to tackle this problem. – Kamal

insert_many appears to be much faster than create but skips validations (Mongoid 8.1.2). – Pitre

Answer 4

Mongoid's Model.create method can accept an array to create documents.

From the Mongoid docs:

Person.create([
  { first_name: "Heinrich", last_name: "Heine" },
  { first_name: "Willy", last_name: "Brandt" }
])

https://docs.mongodb.org/ecosystem/tutorial/mongoid-persistence/

Surratt

That still creates them one by one :( – Rakehell