How to let syslog workable in docker?

Question

My application will send out syslog local0 messages. When I move my application into docker, I found it is difficult to show the syslog.

I've tried to run docker as --log-dirver as syslog or journald, both works strange, the /var/log/local0.log show console output of docker container instead of my application's syslog when I try to run this command inside container

logger -p local0.info -t a message

So, I try to install syslog-ng inside the docker container. The outside docker box is Arch Linux (kernel 4.14.8 + systemctl). The docker container is running as CentOS 6. If I install syslog-ng inside the container and start it, it shows following message.

# yum install -y syslog-ng  # this will install syslog-ng 3.2.5
# /etc/init.d/syslog-ng start
Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql'
Starting syslog-ng: Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql'
Error opening file for reading; filename='/proc/kmsg', error='Operation not permitted (1)'
Error initializing source driver; source='s_sys', id='s_sys#0'
Error initializing message pipeline;

Edelweiss

We have a series of blogposts about running syslog-ng in docker that might be useful: syslog-ng.com/blog/central-log-server-docker – Overweight

The right answer is on this page: stackoverflow.com/a/48266312 syslog is based on a network of syslog servers. syslog server becomes a relay if it does "collect and pass". you place one on your laptop or on your VM or "near" if it is in a serverless hosted container. also. people do not understand syslog is everywhere. syslog messages flow all the time all over this planet. – Fruit

Answer 1

CentOS 6:

1.

Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' 
Starting syslog-ng: Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql'

You can fix above error by installing syslog-ng-libdbi package:

yum install -y syslog-ng-libdbi

2.

Error opening file for reading; filename='/proc/kmsg', error='Operation not permitted (1)'
Error initializing source driver; source='s_sys', id='s_sys#0'
Error initializing message pipeline;

Since syslog-ng doesn't have direct access on the kernel messages, you need to disable (comment) that in its configuration:

sed -i 's|file ("/proc/kmsg"|#file ("/proc/kmsg"|g' /etc/syslog-ng/syslog-ng.conf

CentOS 7:

1.

Error opening file for reading; filename='/proc/kmsg', error='Operation not permitted (1)'

The system() source is in default configuration. This source reads platform-specific sources automatically, and reads /dev/kmsg on Linux if the kernel is version 3.5 or newer. So, we need to disable (comment) system() source in configuration file:

sed -i 's/system()/# system()/g' /etc/syslog-ng/syslog-ng.conf

2. When we start it in foreground mode syslog-ng -F we get the following:

# syslog-ng -F
syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted'

So, we need to run syslog-ng as root, without capability-support:

syslog-ng --no-caps -F

Theme

for CentOS 7, this method can not work, because /proc/kmsg is not exist in /etc/syslog-ng/syslog-ng.conf – Edelweiss

ref: #51551124, the major point is replace system() to unix-stream("/dev/log") – Edelweiss

Answer 2

I also had problems getting the standard "syslog" output from my app after it has been dockerized.

I have attacked the problem from a different direction. I wanted to get the container syslogs on the host /var/log/syslog

I have ran my container with an extra mount the /dev/log device and voila it worked like a charm.

docker run -v /dev/log:/dev/log  sysloggingapp:latest

Homing

it works!) thanks for idea to mount host's /dev/log to container's /dev/log – Vitiate

Answer 3

Another way is to set up central logging with syslog/ rsyslog server, then use the syslog docker driver for logging. The syntax to use on the docker run command line is:

$ docker run --log-driver=syslog \
--log-opt syslog-address=udp://address:port image-name

Destination syslog server protocol can be udp or tcp and the server address can be a remote server, VM, a different container or local container address.

Replace image-name with your application docker image name.

A ready rsyslog docker image is available on https://github.com/jumanjihouse/docker-rsyslog

References: Docker Logging at docker.com,

Docker CLI, https://www.aquasec.com/wiki/display/containers/Docker+Containers+vs.+Virtual+Machines

Wiggins

Answer 4

For anyone trying to figure this out in the future for C/C++ programs,

The best way I've found is to just set LOG_PERROR flag in openlog(). That way, your syslog will print to stderr, which docker will then log by default (you don't need to run syslog process in docker for this). This is much easier then trying to figure out how to run a syslog process alongside your application inside your docker container (which docker probably isn't designed to do anyway).

Higgs

how does one do this? Would you mind elaborating? Edit: seems this is a php thing and doesn't suit my use case where I'm not using a php app in the container. – Migraine