Executing mongoimport inside code with Javascript/Node.js

Question

Is there any library available in node.js/javascript that allows an individual to use mongoimport in code?

To my understanding, mongoimport is kinda like an .exe, which you have to execute it first before being able to use its text input environment.

Is it possible to execute mongoimport in my code and then parse whatever commands I need directly in my code?

My current algorithm involves:

fs.appendFile('log.txt',JSON.stringify(obj, null, 2));

obj is an object which specifies what functions to parse into JSON.stringify with the res method of node.js (which requests HTTP responses)

var obj = {};
obj.url = hostNames[i];
obj.statusCode = res.statusCode;
obj.headers = res.headers;

Then I use mongoimport to import this JSON doc into my MongoDB.

mongoimport --host localhost -db scrapeapp -collection scrape --file log.txt --jsonArray

This method is obviously inefficient. I would like to do all these steps in one going.

Help appreciated

Friedland

Have you looked at the MongoDB node.js driver? This is what you want to use if you would like to talk directly from a node.js service to MongoDB. Documentation is here: docs.mongodb.org/ecosystem/drivers/node-js – Translator

To extend what James said, I think you have something of a mis-understanding over what mongoimport is used for. Most of your interactions with mongodb will occur through a driver. The driver allows you to insert, update, and delete documents (objects) in your database through a specific language, in this case javascript. Mongoimport should not be used for normal insertions in your db, it is mainly used for importing data that has been exported from other sources, like other databases or applications. – Osteoma

You are right. I do currently have a misunderstanding with the use of MongoDB. I am currently scraping headers from a couple (thousand) sites, which I will then do analysis of in a couple month's time. I will change my algorithm to do an insert() header by header instead of doing an import. What do you think? db.scrape.insert(JSON.stringify(obj,null,2)); – Friedland

Yea, you need to be doing insert :) – Farfetched

Answer 1

This is how I do it in my code

let exec = require('child_process').exec
let command = 'mongoimport -d database -c collection --file import.json'
exec(command, (err, stdout, stderr) => {
  // check for errors or if it was succesfuly
  cb()
})

I exec the mongoimport command and then I do pass the cb next the code to be accesible, or if you do not use an asynchronous style you can do it synchronously with child_process.execSync(command[,options])

Renfred

Answer 2

I'm in no way a node expert - but if you have existing JSON files, you could execute mongoimport in Node as shell command as described here or in various answers.

Boroughenglish

He said he is aware of that. He's asking how to do the equivalent of mongoimport in JavaScript instead of the mongo shell. – Noami

He asked "Is it possible to execute mongoimport in my code". The first article I linked to is titled "Execute A Unix Command With Node.js" - so technically, this would work. But I'm aware that this is probably not what he had in mind in terms of efficiency. – Boroughenglish