MongoDB log file growth

Question

Currently my log file sits at 32 meg. Did I miss an option that would split the log file as it grows?

Animatism

Mine grew to 23GB after being left unattended for a year and blew up the server :O – Dromond

Guys, that's nothing. My whole production server went down because this file was ... 345Gig. I was speechless – Arondell

Answer 1

Rotate the logs yourself

http://www.mongodb.org/display/DOCS/Logging

or use 'logrotate' with an appropriate configuration.

Jailhouse

"with an appropriate configuration" : that is exactly the question... What is an appropriate configuration. Your answer (and MongoDB documentation) are not clear about this subject. – Hileman

Answer 2

You can use logrotate to do this job for you.

Put this in /etc/logrotate.d/mongod (assuming you use Linux and have logrotate installed):

/var/log/mongo/*.log {
    daily
    rotate 30
    compress
    dateext
    missingok
    notifempty
    sharedscripts
    copytruncate
    postrotate
        /bin/kill -SIGUSR1 `cat /var/lib/mongo/mongod.lock 2> /dev/null` 2> /dev/null || true
    endscript
}

Exophthalmos

Just FYI, I installed from 10gen's official rpms and lograte was configured automatically. – Sexagesima

Useful to know. The installations I've previously done didn't have that. – Exophthalmos

Thanks for the config file. Be sure to check the paths in this script, though - in my case that was /var/log/mongodb/ and /var/lib/mongodb . – Unholy

What is happening in the postrotate. Is this something I should be wary of implementing on a production system? – Playoff

@Matthew: check the docs (linked in the accepted answer). Short version: it causes mongodb to close the previous log and open the next one. Due to Unix file semantics, if you don't do that, mongo will keep writing to the previous file. – Enterovirus

The above logrotate script needs copytruncate to avoid crashing MongoDB versions 2.4.7 and 2.4.6. See jira.mongodb.org/browse/SERVER-11087 for details. – Spathic

Thanks. This sample script along with logrotate man page was very useful and educating for me. Also, as pointed out by @ErwinWessels people must pay attention to the paths. – Calv

May i know the meaning of /bin/kill -SIGUSR1 cat /var/lib/mongo/mongod.lock 2> /dev/null 2> /dev/null || true – Decastyle

It sends a signal to the mongod process to tell it to close and reopen the log file. See here for more on signals: #5351365 – Exophthalmos

I dun know why there are two logs generated, mongo.log-20150511 and mongo.log.2015-05-11T20-02-03 and the second log has no data. since my data is in /export/home/d17/test/data/v1/mongo/repl0 so my postrotate script is /bin/kill -SIGUSR1 cat /export/home/d17/test/data/v1/mongo/repl0/mongod.lock 2> /dev/null 2> /dev/null || true – Decastyle

I only need mongo.log-20150511, but why is mongo.log.2015-05-11T20-02-03 also generated? – Decastyle

@Decastyle - it's because there's 2 log rotation managers. First, logrotate runs, copying mongod.log to mongod.log-20150511. After that, the postrotate script is executed, and when mongod receives SIGUSR1 it also does a rotation, copying mongod.log (now empty) is then copied by mongod (not logrotate) to mongod.log.2015-05-11T20-02-03. You could add a line to your postrotate script to delete all mongod style rotated logs, after the kill command: rm -f /path/to/log/mongo.log*T* – Diao

p.s. you can control logrotate's naming pattern by using the dateformat parameter - see the logrotate manual for more info. linux.die.net/man/8/logrotate – Diao

Great answer. Couple notes. Don't send a SIGUSR1 to Monogo after moving its active file, it will coredump. Not that your config does that, but I did it while testing. Also this worked for me /bin/cat /var/run/mongodb/mongod.pid |xargs kill -SIGUSR1 2> /dev/null|| true Just a little more straight forward in my mind. – Courtesan

@Diao There is a systemLog.logRotate property that you can use which supports logrotate and doesn't create the mongod.log.2015-05-11T20-02-03. Set logRotate to reopen and logAppend to true. See docs.mongodb.com/v3.4/tutorial/rotate-log-files – Theresiatheresina

Answer 3

If you think that 32 megs is too large for a log file, you may also want to look inside to what it contains.

If the logs seem mostly harmless ("open connection", "close connection"), then you may want to start mongod with the --quiet switch. This will reduce some of the more verbose logging.

Mensal

Or, as of 2.0, add quiet = true to your config file (/etc/mongod.conf on some systems). – Evan

Answer 4

Using logrotate is a good option. while, it will generate 2 log files that fmchan commented, and you will have to follow Brett's suggestion to "add a line to your postrotate script to delete all mongod style rotated logs".

Also copytruncate is not the best option. There is always a window between copy and truncate. Some mongod logs may get lost. Could check logrotate man page or refer to this copytruncate discussion.

Just provide one more option. You could write a script that sends the rotate signal to mongod and remove the old log files. mongologrotate.sh is a simple reference script that I have written. You could write a simple cron job or script to call it periodically like every 30 minutes.

Crofton

Answer 5

For Linux and Unix-based systems, you can use the SIGUSR1 signal to rotate the logs for a single process.

For example, if a running mongod instance has a process ID (PID) of 2200, the following command rotates the log file for that instance on Linux:

kill -SIGUSR1 2200

Putsch