How to ignore the exception which are generated during the logging in python?

Question

I am working in a big python module(Python 2.6.6) and doing logging in thousand of places and i am using logger module of python. Now, i want to ignore the exception, which are generated during the logging. One of the basic scenario which could be possible is

import glob
import logging
import logging.handlers

LOG_FILENAME = '/home/abc/logging_rotatingfile_example.out'
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)

# Add the log message handler to the logger
handler = logging.handlers.RotatingFileHandler(
              LOG_FILENAME, maxBytes=20, backupCount=5)
my_logger.addHandler(handler)
# Log some messages
for i in range(20):
    my_logger.debug('i = %d' % i)

Now suppose, during the my_logger.debug('i = %d' % i) some how the file is deleted from the system or permissions are changed and logger couldn't able to log due to disk failure, disk full etc. on the log file and generate the exception on the screen. It is distributed system and n number of process are using logger module. Please note process should not be aborted due to logging exception(due to disk failure, disk full etc.). So i want that exception to be ignored.

NOTE: Now one of the way i know to make wrapper class on python logger and use try-except for the logger function. But i have different requirements and i dont want to use that way so ,is there any other way to do this or any attribute of python logger which can be useful for my condition?

Embowed

Why no try-except? – Cyclonite

You can try wrapping logging with your own class, still you will need to use try-except to ignore exceptions – Sundry

Answer 1

You do not need to ignore exceptions, because the logging module already handles these for you.

All file operations (log emitting, rotation, etc.) are guarded with try..except Exception blocks. If exceptions occur, the logging.Handler.handeError() method will write the exception to sys.stderr instead. You can set logging.raiseExceptions to False to silence these:

import logging

logging.raiseExceptions = False

Quoting the documentation:

This method should be called from handlers when an exception is encountered during an emit() call. If the module-level attribute raiseExceptions is False, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The specified record is the one which was being processed when the exception occurred. (The default value of raiseExceptions is True, as that is more useful during development).

If the filename you passed to RotatingFileHandler cannot be created, then the exception raised doesn't happen during logging however, it takes place while creating the handler. You'll either have to handle that exception there and then, or set delay=True when creating the handler to postpone opening the file until logging time:

handler = logging.handlers.RotatingFileHandler(
    LOG_FILENAME, maxBytes=20, backupCount=5, delay=True)

What you use comes down to wether or not you see providing an incorrect path in LOG_FILENAME is an application error or not at that point.

Kinlaw

Please see the updated question,i have also tried your way but still result is same. – Embowed

@ShanuPandit: that exception is generated while setting up the handler, not during logging. Catch that exception and use a different handler instead. – Kinlaw

@ShanuPandit: alternatively, set delay to true to not open the file immediately. – Kinlaw

@MartijinPieters, thanks for your suggestion. Now i understand the purpose of raiseExceptions Just for another scenario, please see the updated section in question, Now if i delete the file manually during the sleep, so it should generate the exception at my_logger.debug('i = %d' % i), but it is not throwing any exception, even raiseExceptions is set. – Embowed

@ShanuPandit: on POSIX platforms deleting the file only removes the directory entry; any programs still holding an open file object can still write to the disk. Deleting the file won't cause an error. – Kinlaw

@ShanuPandit: once the file is closed, the on-disk data will be reaped then. – Kinlaw

@MartijinPieters. thank you for very useful information. But as you said any object which is handling the open file still can write on disk even file is deleted, but what if disk is failed or disk is full? – Embowed

@ShanuPandit: then an IOError exception will be raised and that'll be handled by the Handler.handleError() method. – Kinlaw

@MartijinPieters. so which means, in that case unset the raiseExceptions won't work. So do you have any document or live program which uses Handler.handleError() which suites best to mys situation? – Embowed

@ShanuPandit: no, it'll work just fine. What gave you the idea it won't work? I have no idea what is special about your situation, and I don't have any examples, sorry. – Kinlaw

@MartijinPieters,you only said that,Handler.handleError() will handle it. but i think i understand what you meant to say. If disk fail or disk full happens then exception will be handled by Handler.handleError() but as raiseExceptions is unset, it will silently ignored the exception. please correct me if i still understand wrongly. – Embowed

Yes, you understand correctly. If raiseExceptions is set to False (not unset), the exception will be ignored. – Kinlaw

@MartijinPieters, OK now i am understood, thanks for your help. – Embowed

@MartijinPieters, thanks for your help, but after using your approach, i was working on my project, but still exception is raised even raiseExceptions is set to False. I really need some guidance for this. Please see the updated section in question which i got as traceback. – Embowed

@ShanuPandit: please consider asking a new question for new issues. I can't reproduce your code on Python 2.6.9. What specific version of Python is this with (run python -V)? Even with the initial version 2.6 the IOError exception is caught and handled, however. – Kinlaw

@ShanuPandit: the line numbers suggest 2.6.6 or newer, but you can only see that traceback printed if raiseExceptions is set to True. The shortness of the traceback certainly suggests that the message you see is printed by Handler.handleError(). The traceback would be longer otherwise. – Kinlaw

@ShanuPandit: in other words, you do not have raiseExceptions set to False. – Kinlaw

@MartijinPieters, thanks for your always help,Sir, i did set the raiseExceptions to False, and i printed the value of it in both file, first one from where i am logging and getting traceback, here thte value is False and the second one in python2.6/logging/__init__.py but here the value is True. could it be possible that some how my changes are not reflecting in python2.6/logging/__init__.py ? – Embowed

@ShanuPandit: without seeing your code, how you set raiseExceptions to False and the overall structure of your approach, that's impossible to answer. If you do from logging import raiseExceptions, then raiseExceptions = False, you are not setting the logging module global, for example. Use import logging then logging.raiseExceptions = False. When running multiple, separate Python interpreters, do this for each and every interpreter. – Kinlaw

Answer 2

So after your edit this is not really valid.

You can use the try - except method

Example:

try:
    file_handler =open("not_a_file_for_sure","r")
except IOError:
    print "File not found"

You can except any kind of error, or even all of them, but that is not advised.

Just except the error you are getting and continue or pass it if you don't care about it.

Sacramentalist

This is of little use to I/O errors during logging however, as those don't raise an exception. The logging module handles those instead. – Kinlaw