PM2 doesn't log Python3 print statements

Question

I'm using PM2 to run a Python program in the background like so

pm2 start helloworld.py

and it works perfectly fine. However, within helloworld.py I have several print statements that act as logs. For example, when a network request comes in or if a database value is updated. When I run helloworld.py like so:

python3 helloworld.py

all these print statements are visible and I can debug my application. However, when running

pm2 logs helloworld

none of these print statements show up.

Mccartney

Answer 1

This question is a few months old, so maybe you figured this out a while ago, but it was one of the top google hits when I was having the same problem so I thought I'd add what I found.

Seems like it's an issue with how python buffers sys.stdout. In some platforms/instances, when called by say pm2 or nohup, the sys.stdout stream may not get flushed until the process exits. Passing the "-u" argument to the python interpreter stops it from buffering sys.stdout. In the process.json for pm2 I added "interpreter_args": "-u" and I'm getting logs normally now.

Bubbler

I couldn't get interpreter_args working as a pm2 command line option. So I made a bash script that calls python -u myscript.py and pm2 started that bash script. – Ocreate

python -u flag worked! – Fluoridation

Answer 2

For those who are still having this issue:

"python -u .../your_script" instead of "python .../your_script" is how you designate that you don't want the interpreter to flush the logs, but for me, it was still only outputting the last 15 lines after the script finished (which for my server was never).

Turns out there was a zombie daemon process still running things without the "-u" added. I went into my computer processes and force-quit every process having to do with pm2 or daemon and it fixed the issue when I re-ran my dev scripts.

In my case, the zombie process came right after we refactored and moved around the filetree, so the old daemon process wasn't accessed and killed properly.

Gestation

Answer 3

Check the folder #HOME/.pm2/logs

See for example the folder structure section here: http://pm2.keymetrics.io/docs/usage/quick-start/

Also consider using a configuration file with an explicit logs folder that is relative to your scripts. (Note this folder must exist before pm2 can use it.) See http://pm2.keymetrics.io/docs/usage/application-declaration/

{
"apps": [
    {
        "script": "app/server.js",
        "log_date_format": "YYYY-MM-DD HH:mm Z",
        "error_file": "logs/server.web.error.log",
        "out_file": "logs/server.web.out.log",
    ...

Nice way to follow these log files is to run tail

tail -f logs/*.log

UPDATE: To be clear, using a configuration file works for python scripts. Just create a json configuration file that specifies your script and where you want the output to go. For example

{
  "apps": [
    {
      "name": "Test Python",
      "script": "test.py",
      "out_file": "test.out.log",
    }
  ]
}

Then run it pm2 start test.json Look for the process id in the results. Use this process ID to stop your process and to see where the log file is. E.g. pm2 show 3

Yankee

Your example references a Node project, while the issue is about Python. – Debus

@DesignbyAdrian the answer above works for python scripts. I will add more details. – Yankee

@DesignbyAdrian I appreciate the down vote was appropriate because I didn't address the original poster's entire question. Could you please consider removing the down vote? Thanks – Yankee