systemctl service systemd-notify not working with non-root user

Question

I have a simple example of a service unit and bash script on Red Hat Enterprise Linux 7 using Type=notify that I am trying to get working.

When the service unit is configured to start the script as root, things work as expected. When adding User=testuser it fails. While the script initially starts (as seen on process list) the systemctl service never receives the notify message indicating ready so it hangs and eventually times out.

[Unit]
Description=My Test
[Service]
Type=notify
User=testuser
ExecStart=/home/iatf/test.sh
[Install]
WantedBy=multi-user.target

Test.sh (owned by testuser with execute permission)

#!/bin/bash

systemd-notify --status="Starting..."
sleep 5
systemd-notify --ready --status="Started"

while [ 1 ] ; do
  systemd-notify --status="Processing..."
  sleep 3
  systemd-notify --status="Waiting..."
  sleep 3
done

When run as root systemctl status test displays the correct status and status messages as sent from my test.sh bash script. When User=testuser the service hangs and then timesout and journalctl -xe reports:

Jul 15 13:37:25 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7193.
Jul 15 13:37:28 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7290.
Jul 15 13:37:31 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7388.
Jul 15 13:37:34 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7480.

I am not sure what those PIDs are as they do not appear on ps -ef list

Never

Answer 1

This appears to be known limitation in the notify service type

From a pull request to the systemd man pages

    Due to current limitations of the Linux kernel and the systemd, this
    command requires CAP_SYS_ADMIN privileges to work
    reliably. I.e. it's useful only in shell scripts running as a root
    user.

I've attempted some hacky workarounds with sudo and friends but they won't work as systemd - generally failing with

No status data could be sent: $NOTIFY_SOCKET was not set

This refers to the socket that systemd-notify is trying to send data to - its defined in the service environment but I could not get it reliably exposed to a sudo environment

You could also try using a Python workaround described here

python -c "import systemd.daemon, time; systemd.daemon.notify('READY=1'); time.sleep(5)"

Its basically just a sleep which is not reliable and the whole point of using notify is reliable services.

In my case - I just refactored to use root as the user - with the actual service as a child under the main service with the desired user

Coypu

Cannot find this in the man page anymore. Has this been resolved? – Blevins

@Blevins my research last night indicates that this was fixed in systemd 246 (which hasn't landed in ubuntu 20.04 though). In particular, this PR, which fixes the issue, first landed in 246-rc1. – Mast

@Mast Very likely chance that systemd 246 won't ever land in 20.04 - given systemd is the init system its basically locked in for the lifetime of the OS. If this is something you want I'd be looking at newer operating systems – Coypu

I do not understand why there is time.sleep(5) in python workaround. I removed it and I noticed that sometimes notification does not work because timeout occurs. I am confusing if removing sleep is responsible for it. Is sleep really necessary? – Terror

Answer 2

sudo -u USERACCOUNT_LOGGED notify-send "hello"

Flathead

Answer 3

You can add AmbientCapabilities=CAP_SYS_ADMIN to the unit file:

[Service]
Type=notify
AmbientCapabilities=CAP_SYS_ADMIN
User=someUser

Also, to avoid race-condition issues, the service should send BARRIER=1 after READY=1.

See https://man.archlinux.org/man/sd_notify_barrier.3.en

---

Note: If SELinux (Mostly for RedHat OS) is enabled:

root# getenforce
Enforcing

Add systemd_notify_t to the permissive types:

semanage permissive -a systemd_notify_t

You can check with:

semanage permissive -l

Rawdin

This isn't relevant to the (now very old) distro and systemd version this question is about, which had a separate (now resolved) bug that prevented correct operation even without SELinux in play. Consider asking a new, SELinux-specific question and posting this as an answer to it. – Maneater