Rails/Mongoid: Parent object not recognising child of has_many/belongs_to relation after mongoimport

Question

A CSV containing the following rows is imported into mongodb via CSV using the mongoimport tool:

object_1_id,field1,field2
52db7f026956b996a0000001,apples,oranges
52db7f026956b996a0000001,pears,plums

The fields are imported into the collection Object2.

After import the rows are confirmed to exist via the console.

#<Object2 _id: 52e0713417bcabcb4d09ad12, _type: nil, field1: "apples", field2: "oranges", object_1_id: "52db7f026956b996a0000001">
#<Object2 _id: 52e0713517bcabcb4d09ad76, _type: nil, field1: "pears", field2: "plums", object_1_id: "52db7f026956b996a0000001">

Object2 can access Object1 via object_1_id:

> o = Object2.first
#<Object2 _id: 52e0713417bcabcb4d09ad12, _type: nil, field1: "apples", field2: "oranges", object_1_id: "52db7f026956b996a0000001">
> o1 = o.object_1
#<Object1 _id: "52db7f026956b996a0000001", other_fields: "text and stuff">

But Object1 cannot see any of the Object2 rows that were imported with mongoimport. It can see all rows that have been created via the console or other means:

> o1.object_2s.count
10
> o1.object_2s.find("52e0713417bcabcb4d09ad12")
Mongoid::Errors::DocumentNotFound:
    Document not found for class Object2 with id(s) 52e0713417bcabcb4d09ad12.

TL;DR Object1 doesn't appear to recognise child models imported via mongoimport, despite the child correctly storing the parent ID and being able to identify its parent.

Poacher

Did your import mess up the types perhaps? Note that the Object1 ids are showing up in quotes (suggesting that they're strings) whereas the Object2 id appears without quotes (suggesting that it is a BSON ObjectId). Check from the MongoDB shell and see if you get "52db7f026956b996a0000001" or ObjectId("52db7f026956b996a0000001"). – Hyo

Answer 1

As per mu is too short's comment the ids were being imported as Strings instead of BSON ObjectIds.

mongoexport and mongoimport (I was only using the latter) only support strings and numbers (See: https://mcmap.net/q/627705/-how-to-import-mongodb-objectid-from-csv-file-using-mongoimport).

In order to import data with type from a CSV you have to use Extended JSON dumps as explained in the above link.

Quick and dirty solution:

1) Export the collection you want to import as JSON using mongoexport:

mongoexport -d database -c collection -o output.json

2) Grab the first line of the export file. It should look something like this:

{ "_id" : { "$oid" : "52dfe0106956b9ee6e0016d8" }, "column2" : "oranges", "column1" : "apples", "object_1_id" : { "$oid" : "52dfe0106956b9ee6e0016d8" }, "updated_at" : { "$date" : 1390403600994 }, "created_at" : { "$date" : 1390403600994 } }

3) The _id field as well as any other fields you don't want to import.

4) Use your language of choice to generate a JSON file using the JSON snippet as a template for each line.

5) Import the new JSON file using mongoimport:

mongoimport -d database -c collection --type json --file modified.json

This will preserve types better than CSV is capable. I'm not sure whether it is as reliable as using mongodump and mongorestore but they aren't an option for me since my CSV file comes from elsewhere.

Poacher