Why can't I stop my Mojo app using systemd?

Question

I can fire up my Mojo app via systemd without any issues, but I can't stop or restart it in the same way. I've poached my config mostly from the Mojo docs at http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Pre-forking:

[Unit]
Description=ldsquiz.org
After=network.target

[Service]
Type=simple
Restart=always
RestartSec=30
ExecStart=/home/sites/www.ldsquiz.org/checkout/bin/carton-exec app.pl prefork -m production -l http://*:5103
KillMode=process
User=ldsquiz
Group=ldsquiz
SyslogIdentifier=ldsquiz-org

[Install]
WantedBy=multi-user.target 

If I stop it via sudo service ldsquiz-org stop then syslog says Jan 13 16:37:48 web-hosting systemd[1]: Stopped ldsquiz.org. but the processes are still running.

A restart via systemd looks like I would expect, since the listen socket is still in use because of the failed shutdown:

Jan 13 16:39:13 web-hosting systemd[1]: Stopped ldsquiz.org.
Jan 13 16:39:13 web-hosting systemd[1]: Started ldsquiz.org.
Jan 13 16:39:14 web-hosting ldsquiz-org[20353]: Can't create listen socket: Address already in use at 
/home/sites/www.ldsquiz.org/checkout/local/lib/perl5/Mojo/IOLoop.pm line 126.
Jan 13 16:39:14 web-hosting systemd[1]: ldsquiz-org.service: Main process exited, code=exited, status=98/n/a
Jan 13 16:39:14 web-hosting systemd[1]: ldsquiz-org.service: Unit entered failed state.
Jan 13 16:39:14 web-hosting systemd[1]: ldsquiz-org.service: Failed with result 'exit-code'.
Jan 13 16:39:15 web-hosting ldsquiz-org[16886]: [Sat Jan 13 16:39:15 2018] [info] Creating process id file "/tmp/prefork.pid"

What am I missing here?

Richly

I usually control my systemd units with systemctl start/stop/status/restart $serviceName. Does that make any difference? I.e. systemctl stop ldsquiz-org? – Jempty

@Jempty that seems to have the same problem for me. It thinks that the app has been stopped but the original processes are still hanging around. – Richly

I know the docs you refer to say Type=simple but this prefork parameter makes me doubt. Perhaps give Type=forking a try. See here for the possible values. Perhaps you also need to play a bit with the KillMode= setting. See here for the possible values. And remember to call systemctl daemon-reload whenever you fiddle with the unit file. – Jempty

@PerlDuckwonderful! KillMode=control-group did the trick for me. Looks like I'll need to see about a Mojo doc patch. If you'd like to write an answer I'm happy to accept it. – Richly

As an addendum, this issue has now been addressed in the Mojo docs: github.com/kraih/mojo/commit/… – Richly

Answer 1

Apparently there's a flaw in the Mojolicious Cookbook w.r.t. preforking and systemd units.

They say:

And to manage the pre-forking web server with systemd, you can use a unit configuration file like this.

[Unit]
Description=My Mojolicious application
After=network.target

[Service]
Type=simple
ExecStart=/home/sri/myapp/script/my_app prefork -m production -l http://*:8080
KillMode=process

[Install]
WantedBy=multi-user.target

The KillMode=process setting turns out to be wrong and needs to be replaced with KillMode=control-group — which is the default setting, so we can drop it altogether.

The KillMode docs say:

If set to control-group, all remaining processes in the control group of this unit will be killed on unit stop (for services: after the stop command is executed, as configured with ExecStop=). If set to process, only the main process itself is killed.

^{(emphasis mine)}

Thus:

[Unit]
Description=My Mojolicious application
After=network.target

[Service]
Type=simple
ExecStart=/home/sri/myapp/script/my_app prefork -m production -l http://*:8080
KillMode=control-group

[Install]
WantedBy=multi-user.target

Note that the KillMode=control-group can be omitted as this setting is the default.

References:

• https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillMode=
• https://www.freedesktop.org/software/systemd/man/systemd.service.html#Type=

Jempty

Yes. That "KillMode" setting has since been removed from the example docs: github.com/mojolicious/mojo/pull/1183 – Lashing