How to rotate log files while running server with pm2?

Question

We are using node module pm2 to run the server and capturing application logs.

But as the traffic is very huge, huge data is getting stored in single file which are around more than 100Gb.

Is there any possibility that we change the file every 1 hour or every 1Gb file without restarting server?

Currently we are manually doing this, restarting server and renaming the existing file which is creating issue.

Xanthe

Answer 1

You can use pm2-logrotate

pm2 install pm2-logrotate

# max log size is 1GB
pm2 set pm2-logrotate:max_size 1G

# compress logs when rotated (optional)
pm2 set pm2-logrotate:compress true

# force rotate every hours
pm2 set pm2-logrotate:rotateInterval '0 * * * *'

Glauce

then how to see the log ? after rotate its not pointing to a new log ? ?? – Christos

Answer 2

I personally do not use pm2-logrotate because it is no longer maintained, and even worse, it is quite buggy. One time I used it on my production server, and it immediately and repeatedly created large log files leading to 0% space and crashed the server. That was not a good day.

PM2 website has a section called "Setting up a native logrotate" wherein it tells you to run:

sudo pm2 logrotate -u user

Obviously change user to the actual user that is running pm2. It will create a file at /etc/logrotate.d/pm2-user and edit it so that it looks like this:

/home/user/.pm2/pm2.log /home/user/.pm2/logs/*.log {
        su user user
        rotate 12
        weekly
        missingok
        notifempty
        compress
        delaycompress
        copytruncate
        create 0640 user user
}

The most important part is su user user part. Logrotate runs as root, and it does not like creating log files that non-root users can view, so it will fail, often times silently. And this runs against npm philosophy, which prefers to run things generally without sudo privileges. By specifying the user in the logrotate config file, you get around this problem.

It took me a while to figure this out - hope it helps someone.

Lennalennard

hey @Lennalennard thanks for this informative posting. But - I can't find the section you refer to on the PM2 website (well not anymore, it might have been removed since you posted I guess), and as a result though pm2 still has "logrotate" option it is effectively undocumented! And the PM2 website just talks about using pm2-logrotate (still, even though I admit it doesn't look maintained any longer). I wanted to lookup the docs for all the options you've listed in your posting, to see what I might want to change for my implementation. thanks! – Charwoman

Hi @AndyLorenz, here is the section link: pm2.keymetrics.io/docs/usage/log-management/… – Pamilapammi

Following what I believe to be standard pm2 practice, on my installation pm2 runs as a non-privileged user. There is no pm2 installed for the root user, and therefore the attempted sudo fails. – Aggarwal

My understanding is that you are supposed to install pm2 with sudo privileges, because in the installation documentation, they ask you to npm install pm2 -g and the -g (global flag) typically will not work without sudo. – Lennalennard