Does python logging.handlers.RotatingFileHandler allow creation of a group writable log file?

Question

I'm using the standard python (2.5.2) logging module, specifically the RotatingFileHandler, on a linux system. My application supports both a command-line interface and a web-service interface. I would like to have both write to the same log file. However, when the log file gets rotated, the new file has 644 permissions and is owned by the web server user which prevents the command-line user from writing to it. Can I specify that new log files should be group-writable in the logging configuration or during logging initialization?

I have looked into the mode setting (r/w/a), but it doesn't seem to support any file permissions.

Telles

Answer 1

I resorted to scanning the logging.handlers module and was unable to see any way to specify a different file permissions mode. So, I have a solution now based on extending the RotatingFileHandler as a custom handler. It was fairly painless, once I found some nice references to creating one. The code for the custom handler is below.

class GroupWriteRotatingFileHandler(handlers.RotatingFileHandler):

    def doRollover(self):
        """
        Override base class method to make the new log file group writable.
        """
        # Rotate the file first.
        handlers.RotatingFileHandler.doRollover(self)

        # Add group write to the current permissions.
        currMode = os.stat(self.baseFilename).st_mode
        os.chmod(self.baseFilename, currMode | stat.S_IWGRP)

I also discovered that to reference the custom handler from a logging config file, I had to bind my module to the logging namespace. Simple to do, but annoying.

from mynamespace.logging import custom_handlers
logging.custom_handlers = custom_handlers

References I found useful: binding custom handlers and creating custom handlers

Telles

I was just about to post the same solution :P – Fallen

One thing missing from this solution is doing the chmod on the creation of log file the first time it is created. – Telles

How do you fix that? The first time file creation – Plush

for fixing the creation time permissions, may be override _open as in @Benempt 's answer – Malvin

to update permissions/ownership of the logfile when it is first created, you can add the relevant os operations inside of an __init__() method after first calling super() (see answer by @Staley below) – Crawley

Answer 2

Here is a slightly better solution. this overrides the _open method that is used. setting the umask before creating then returning it back to what it was.

class GroupWriteRotatingFileHandler(logging.handlers.RotatingFileHandler):    
    def _open(self):
        prevumask=os.umask(0o002)
        #os.fdopen(os.open('/path/to/file', os.O_WRONLY, 0600))
        rtv=logging.handlers.RotatingFileHandler._open(self)
        os.umask(prevumask)
        return rtv

Benempt

What if the application crashes while opening ? Does the umask stay set with 0o002 – Alvaroalveolar

@BenjaminToueg I think it's specific to that process and doesn't affect the system default umask. – Ululant

Answer 3

Here is a simple solution worked fine:

import os

class GroupWriteRotatingFileHandler(handlers.RotatingFileHandler):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        os.chmod(self.baseFilename, 0o0777)  # You can change whatever permission you want here.

        # you can also change the group of the file:
        os.chown(self.baseFilename, uid, gid)   # pass the user_id and group_id you want to set 

Staley

It seems that doesn't work. If change the log files to be owned by root, and give all of the the permission of "0o0777", it throws ValueError: Unable to configure handler 'some_handler': [Errno 1] Operation not permitted: . It looks that the process owner needs to have the ownership of the log files. – Diabolism

I think this is only the permission issue. in your case, you might need sudo to change the file permission, you can test it with command-line first. if so, you can change os.chmod, os.chown two lines to a sudo version code. – Staley

However the ValueError: Unable to configure handler 'some_handler': [Errno 1] Operation not permitted: was gone, if using handlers.RotatingFileHandler, and the log file with the permission of 0o0666 – Diabolism

Answer 4

Here's a complete solution for Django based on rob's solution. in my_module :

import logging
import logging.handlers
import os

The class= that happens during the logging configuration is evaluated in the namespace of the logging module, and by default this does not have a binding to handlers. So we have to put it in explicitly before we can extend it. See this SO article

logging.handlers = logging.handlers

It's this magical incantation that took me forever to find - I couldn't believe it did anything! Finally, Jon's class will load without errors.

class GroupWriteRotatingFileHandler(logging.handlers.RotatingFileHandler):    
    def _open(self):
        prevumask = os.umask(0o002)
        rtv = logging.handlers.RotatingFileHandler._open(self)
        os.umask(prevumask)
        return rtv

To use this for Django add the following this in the settings file

from my_module import GroupWriteRotatingFileHandler
logging.handlers.GroupWriteRotatingFileHandler = GroupWriteRotatingFileHandler

And then in LOGGING['handlers']['file'] you have

'class': 'logging.handlers.GroupWriteRotatingFileHandler'

Rainie

Answer 5

James Gardner has written a handler that only rotates files, not creating or deleting them: http://packages.python.org/logrotate/index.html

Nagging

Answer 6

$ chgrp loggroup logdir
$ chmod g+w logdir
$ chmod g+s logdir
$ usermod -a -G loggroup myuser
$ umask 0002

Monometallic

could you explain or at least give a short comment what the commands do? – Border

Answer 7

It looks that def _open(self): worked with umask(0o000) to get all the permissions of -rw-rw-rw-.

os.chmod(self.baseFilename, 0o0777) failed with ValueError: Unable to configure handler 'some_handler': [Errno 1] Operation not permitted: if the log file has ownership of root:root that's different from running process's, such as testuser.

from logging import handlers

import logging
import os

class GroupWriteRotatingFileHandler(handlers.RotatingFileHandler):
    def _open(self):
        prevumask = os.umask(0o000)  # -rw-rw-rw-
        rtv = logging.handlers.RotatingFileHandler._open(self)
        os.umask(prevumask)
        return rtv

LOGGING = {
    'handlers': {
        'db_handler': {
                'level': 'DEBUG',
                'class': 'log.GroupWriteRotatingFileHandler',
                'filename': PATH_TO_LOGS + '/db.log',
                'maxBytes': maxBytes,
                'backupCount': backupCount,
                'formatter': 'standard',
        },

Log files:

logs]# ls -lrt
-rw-rw-rw- 1 root root 71 Apr  1 16:02 db.log

logs]# ls -lrt
total 0
-rw-rw-rw- 1 testuser testuser 0 Apr  1 16:20 db.log

Diabolism

Answer 8

I think what described here is an anti-pattern - different processes should not write data into the same file.

And non of the solutions above worked for me, causing different permissions issues in different scenarios.

As a temp workaround I've added a random suffix into the log filename so each process will get a unique filename on a startup.

The proper way to solve this issue - have a centralized log handler (log server), e.g. rsyslog.

Pending