MongoDB won't start after server crash

Asked 4/12, 2012 at 10:0 Answered 18/12, 2018 at 10:15

My Ubuntu computer had crashed, and when I restarted it MongoDB wasn't working. I tried the following commands, and got the following output:

$ mongo
Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91
exception: connect failed

$ service mongodb status
mongodb stop/waiting

$ service mongodb restart
stop: Unknown instance: 
start: Rejected send message, 1 matched rules; type="method_call",
       sender=":1.57" (uid=1000 pid=2227 comm="start mongodb ")
       interface="com.ubuntu.Upstart0_6.Job" member="Start" error name="(unset)"
       requested_reply="0"
       destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init")

$ tail /var/log/mongodb/mongodb.log
[initandlisten] exception in initAndListen: 12596 old lock file, terminating
dbexit: 
[initandlisten] shutdown: going to close listening sockets...
[initandlisten] shutdown: going to flush diaglog...
[initandlisten] shutdown: going to close sockets...
[initandlisten] shutdown: waiting for fs preallocator...
[initandlisten] shutdown: closing all files...
[initandlisten] closeAllFiles() finished
dbexit: really exiting now

(Output reformatted to match website layout.)

What happened? How can I fix it?

Unsphere answered 4/12, 2012 at 10:0 Comment(0)

179

The log file is telling you that you have an "old lock file". MongoDB keeps a lock file while it's running. It creates this file when it is started, and deletes it when it's stopped. When the computer crashes (or MongoDB crashes, e.g. via kill), this file is not deleted, and thus the database does not start. The existence of this file indicates unclean shutdown of MongoDB.

Two things can be done:

If this is a development machine and you haven't been using your database (and neither have your programs), you can remove the file manually. For MongoDB 2.2.2 running on Ubuntu 12.10, it's in /var/lib/mongodb/mongod.lock. For other versions, the file could be in a different path or it could be named mongo.lock.
The safer route is to follow MongoDB's Durability and Repair guide. In summary, for a machine with the above configuration, you should execute the following commands:
```
sudo -u mongodb mongod --repair --dbpath /var/lib/mongodb/
sudo service mongod start
```

Unsphere answered 4/12, 2012 at 10:0 Comment(8)

For Windows, see #11878430 – Kami 23/9, 2014 at 15:4

Dear @HosamAly, How can I resolve it, because the server usually crashes. so I must do manually after the server crashes? this is so bad, because others database (mysql, postgresql,..) always start successfully after server crashed – Coon 10/10, 2014 at 4:59

@JohnNguyen You could automate it, but I would advise against that, because you won't get a chance to look at the damage that may be caused by the "repair" process. – Unsphere 11/10, 2014 at 14:49

Does the first line not suppose to be: sudo mongod -u mongodb --repair --dbpath /var/lib/mongodb/ ? – joshy 7 mins ago – Narcosis 15/7, 2015 at 11:8

@Narcosis I don't know whether your version would work. Mine runs mongod with the "mongodb" user permissions. Yours runs it with root permissions. – Unsphere 21/7, 2015 at 15:4

@HosamAly I have the -u so -u mongodb means running with user mongodb, – Narcosis 11/8, 2015 at 14:42

The last line should be: sudo service mongod start (mongod instead of mongodb) – Inbeing 8/2, 2016 at 0:47

depending on your installation sudo is or is not necessary, mongod --dbpath /data/db --repair was enough in my case – Concretize 2/11, 2016 at 7:24

all I had to do was run: sudo mongod --repair

then:

sudo mongod

Louisville answered 29/10, 2015 at 14:7 Comment(1)

sudo mongod means that your database is running as root, which is not recommended from a security point of view. – Unsphere 29/10, 2015 at 21:9

Based on my experience, I usually delete the "mongod.lock" file that is inside the database folder - In my case:

*I browse to where the database is installed on my ubuntu i.e. "data" folder.(cd data); list the files (ls) *Then, I will remove the "mongod.lock" file that was automatically created when the database crashed, by issuing "rm mongod.lock" file.

After which I will either issue "./mongod" to start the mongo deamon or mongo to start the mongo shell. And everything will be fine.

Colorado answered 1/6, 2016 at 10:35 Comment(0)

Check to see if you have enough free space on your server. If there is no room left mongodb won't start.

Curhan answered 31/1, 2015 at 6:34 Comment(1)

This is something good to check, even if it is incorrect in this case. – Adjudicate 28/9, 2015 at 13:24

If you did not use monitoring tools like Bluepill or Monit etc you will have to face this issue because after server crash due to some reason mongo didnot start its daemon automatically then you have to make it work manually, like sudo service mongod restart I figured this issue but it needs some more tasks to be done, please make sure your dbpath at /etc/mongod.conf before starting your mongo daemon.

For me it was

storage:
  dbPath: /var/lib/mongodb

When i enter mongod command it shows me MongoDB starting : pid=10795 port=27017 dbpath=/data/db 64-bit host=xyz.com make sure your dbpath is as same as mentioned in /etc/mongod.conf

To do this you can type sudo mongod --dbpath /var/lib/mongodb and then use mongod command to start your mongo process at your desired dbpath.

FYI: start your mongo process with mongod command

Whisler answered 30/10, 2018 at 4:58 Comment(0)

This is probably not the best solution, but if you're desperate you can try this. It seemed that only the journal was a problem for me, so I took these steps:

Create a new data directory. Possibly /var/lib/mongodb2
Update your mongod.conf to point to the new data dir.
Start mongoDB.
If it starts successfully, then you can shut down mongo again and proceed, otherwise you can stop reading here.
Locate your previous data dir and copy the files for your database(s) to your new data directory (example, admin.0 admin.1 admin.ns, etc)
Start mongoDB again (still using the new data directory)

After completing these steps (took less than 5 min), I was up and running and all data appeared to be ok.

Creed answered 2/12, 2017 at 23:7 Comment(0)

Thank's guys. We also faced an issue where MongoDB was restarting over and over again an it was complaining about old lock file. I stopped MongoDB from Windows service list and then I deleted the mongod.lock file. After that I was able to start MongoDB service correctly and it worked fine.

Enaenable answered 3/10, 2018 at 5:47 Comment(0)

Removing .lock file from mongo data directory dbpath works for me.

e.g sudo sudo rm {data-directory}/mongod.lock

Botany answered 18/12, 2018 at 10:15 Comment(0)

Recommended topics

Hot tags