What is the advantage of using supervisord over monit?

Question

We have a custom setup which has several daemons (web applications + background tasks) running. I am looking at using a service which helps us to monitor those daemons and restart them if their resource consumption exceeds over a level.

I will appreciate any insight on when one is better over the other. As I understand, monit spins up a new process while supervisord starts a sub process. What are the pros and cons of this approach?

I will also be using upstart to monitor monit or supervisord itself. The web application deployment will be done using capistrano.

Ume

Answer 1

I haven't used monit but there are some significant flaws with supervisord.

1. Programs should run in the foreground

   This means you can't just execute /etc/init.d/apache2 start. Most times you can just write a one liner e.g. "source /etc/apache2/envvars && exec /usr/sbin/apache2 -DFOREGROUND" but sometimes you need your own wrapper script. The problem with wrapper scripts is that you end up with two processes, a parent and child. See the the next flaw...

2. supervisord does not manage child processes

   If your program starts child process, supervisord wont detect this. If the parent process dies (or if it's restarted using supervisorctl) the child processes keep running but will be "adopted" by the init process and stay running. This might prevent future invocations of your program running or consume additional resources. The recent config options stopasgroup and killasgroup are supposed to fix this, but didn't work for me.

3. supervisord has no dependency management - see #122

   I recently setup squid with qlproxy. qlproxyd needs to start first otherwise squid can fail. Even though both programs were managed with supervisord there was no way to ensure this. I needed to write a start script for squid that made it wait for the qlproxyd process. Adding the start script resulted in the orphaned process problem described in flaw 2

4. supervisord doesn't allow you to control the delay between startretries

   Sometimes when a process fails to start (or crashes), it's because it can't get access to another resource, possibly due to a network wobble. Supervisor can be set to restart the process a number of times. Between restarts the process will enter a "BACKOFF" state but there's no documentation or control over the duration of the backoff.

In its defence, supervisor does meet our needs 80% of the time. The configuration is sensible and documentation pretty good.

Beestings

Good answer. So, you basically like supervisord but it's not always the right tool for every job. – Sinkhole

I like your answer since you definitely show your experience and tries to set it up to suit your needs (which are probably the same as mine). – Finegan

How is supervisord today with these flaws pr 2020? – Richter

Answer 2

If you want to additionally monitor resources you should settle for monit. In addition to just checking whether a process is running (availability), monit can also perform some checks of resource usage (performance, capacity usage), load levels and even basic security checks (md5sum of a bianry file, config file, etc). It has a rule-based config which is quite easy to comprehend. Also there is a lot of ready to use configs: http://mmonit.com/wiki/Monit/ConfigurationExamples

Monit requires processes to create PID files, which can be a flaw, because if a process does not create pid file you have to create some wrappers around. See http://mmonit.com/wiki/Monit/FAQ#pidfile

Supervisord on the other hand is more bound to a process, it spawns it by itself. It cannot make any resource based checks as monit. It has a nice CLI servicectl and a web GUI though.

Stich

Creating such a wrapper is not really a problem - if you are running a monitoring software, then usually you have some control over your file system. And it only takes to create a trivial script. +1 for good explanation. – Manx

@xavier disagree, a wrapper script is yet anothe SPOF and not every deamon can be wrapped deterministically, think of some java things for example – Stich

@Dārayavahuštdi, you have a valid point, but with supervisord it’s the other way around: some programs like to daemonize, while supervisord requires everything to stay in the foreground. Writing a wrapper for monit seems much more straight forward, though. supervisord.org/subprocess.html#nondaemonizing-of-subprocesses mmonit.com/wiki/Monit/FAQ#pidfile – Scorify

@Dārayavahuštdi in the case of docker containers and keeping them up, docker wants something running in the foreground. In these use cases you're better off with supervisord – Salian

supervisord plugins superlance.readthedocs.org/en/latest can monitor memory ans also it can send out email if the supervisor has restarted it. – Bullshit

Thanks, based on this answer - I tried monit, it worked like charm for my needs to monitor tomcat – Abbott