Running Redis in daemonized form and using Upstart to manage it doesn't work

Question

I've written an Upstart script for Redis as follows:

description "Redis Server"

start on runlevel [2345]
stop on shutdown
expect daemon

exec sudo -u redis /usr/local/bin/redis-server /etc/redis/redis.conf

respawn
respawn limit 10 5

I then configure redis via it's redis.conf to:

daemonize yes

All the documentation and my own experimentation says Redis forks twice in daemonized form and "expect daemon" should work, but the Upstart script is always holding on to the PID of the former parent (PID - 1). Has anyone got this working?

Elmer

I know this is some obscene question necromancy, but... why not use daemonize no and leave the expect clause completely out of your upstart config? (I ask because I'm setting up redis on an ubuntu 14 box) – Butyraldehyde

Answer 1

Other people have the same problem. See this gist.

When the daemonize option is activated, Redis does not check if the process is already a daemon (there is no call to getppid). It systematically forks, but only once. It is somewhat unusual, other daemonization mechanisms may require the initial check on getppid, and fork to be called twice (before and after the setsid call), but on Linux this is not strictly required.

See this faq for more information about daemonization.

Redis daemonize function is extremely simple:

void daemonize(void) {
    int fd;

    if (fork() != 0) exit(0); /* parent exits */
    setsid(); /* create a new session */

    /* Every output goes to /dev/null. If Redis is daemonized but
     * the 'logfile' is set to 'stdout' in the configuration file
     * it will not log at all. */
    if ((fd = open("/dev/null", O_RDWR, 0)) != -1) {
        dup2(fd, STDIN_FILENO);
        dup2(fd, STDOUT_FILENO);
        dup2(fd, STDERR_FILENO);
        if (fd > STDERR_FILENO) close(fd);
    }
}

Upstart documentation says:

expect daemon
Specifies that the job's main process is a daemon, and will fork twice after being run.
init(8) will follow this daemonisation, and will wait for this to occur before running
the job's post-start script or considering the job to be running.
Without this stanza init(8) is unable to supervise daemon processes and will
believe them to have stopped as soon as they daemonise on startup.

expect fork
Specifies that the job's main process will fork once after being run. init(8) will
follow this fork, and will wait for this to occur before running the job's post-start
script or considering the job to be running.
Without this stanza init(8) is unable to supervise forking processes and will believe
them to have stopped as soon as they fork on startup.

So I would either deactivate daemonization on Redis side, either try to use expect fork rather than expect daemon in upstart configuration.

Expugnable

Thanks for the reply. My strace research, as well as normal observation suggests that Redis forks twice. Reading the code, I don't see how that's possible, but somehow it seems to. Also, I did try "expect fork" and that did not result in the correct PID being found. I had seen that Gist, but one can never trust that others have done the in-depth homework to get to the root problem. I assume daemonization is out the question with Upstart. – Elmer

I've just tried a simple strace -f src/redis-server redis.conf 2>strace.log on a daemonized instance, and it does fork only once. Please note on Linux, the clone system call is used to create both processes and threads (and Redis uses a couple of background threads now). The parameters should make it clear if it is a thread or a process. Too bad "expect fork" does not work as we would expect ... – Expugnable

Okay, thanks for confirming it's a single fork. I definitely tried "expect fork" with no success, but I may make another go of it just to make sure. I'll report back. – Elmer

No luck, but I may be doing something wrong. Hopefully someone will figure this out on our behalf. Thanks for the help Didier. – Elmer

Answer 2

The following upstart config seems to be working for me, with upstart 1.5 on ubuntu 12.04, with redis.conf daemonize set to yes:

description "redis server"

start on (local-filesystems and net-device-up IFACE=eth0)
stop on shutdown

setuid redis
setgid redis
expect fork

exec /opt/redis/redis-server /opt/redis/redis.conf

respawn

Interfluent