Logging to two files with different settings

Question

I am already using a basic logging config where all messages across all modules are stored in a single file. However, I need a more complex solution now:

• Two files: the first remains the same.
• The second file should have some custom format.

I have been reading the docs for the module, bu they are very complex for me at the moment. Loggers, handlers...

So, in short:

How to log to two files in Python 3, ie:

import logging
# ...
logging.file1.info('Write this to file 1')
logging.file2.info('Write this to file 2')

Chainey

Answer 1

You can do something like this:

import logging
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')


def setup_logger(name, log_file, level=logging.INFO):
    """To setup as many loggers as you want"""

    handler = logging.FileHandler(log_file)        
    handler.setFormatter(formatter)

    logger = logging.getLogger(name)
    logger.setLevel(level)
    logger.addHandler(handler)

    return logger

# first file logger
logger = setup_logger('first_logger', 'first_logfile.log')
logger.info('This is just info message')

# second file logger
super_logger = setup_logger('second_logger', 'second_logfile.log')
super_logger.error('This is an error message')

def another_method():
   # using logger defined above also works here
   logger.info('Inside method')

Lewin

The default log level is logging.WARNING, so it would be more clear if Logger.setLevel(logging.WARNING) is called. – Archie

Why my logger_1 doesn't output to log file? I've set logger_1.error('error foo'),but still not work – Greatcoat

Combining the comment from @Archie and the question from @Gank... If you want to see the logger_1.info('message_1') line in simplefile_1.log, you need to either set the level to INFO using logger_1.setLevel(logging.INFO), or use logger_1.error('message_1') instead. An INFO message apparently won't get logged when the level is WARNING, which is the default. – Endermic

edited the answer. Set the logger_1.info to logger_1.warning. In this way, the level doesn't have to be set. – Robbi

@Lewin Are logger_1 and logger_2 global? I.e., I can use them inside any function? if not, is it a good idea to make them global in the def main function? if so, how would you go about it? – Kirovograd

@Kirovograd I updated the answer. You could define your loggers after your imports and then use them for the whole module. – Lewin

is this really the only way to do it? I want to call logging.debug(...) or logging.error(...) once and it will go to each file where the threshold lets it through. The Java libraries work this way. – Monologue

@Lewin this does log to file but it also displays on stdout. What changes are reequired if i only want the log in file not on screen ? – Spaetzle

@Spaetzle by default, the logging module doesn't send logged messages to stdout unless a StreamHandler is added to your logging configuration. What type of application do you have? This configuration could be in the application context. – Lewin

Answer 2

def setup_logger(logger_name, log_file, level=logging.INFO):
    l = logging.getLogger(logger_name)
    formatter = logging.Formatter('%(message)s')
    fileHandler = logging.FileHandler(log_file, mode='w')
    fileHandler.setFormatter(formatter)
    streamHandler = logging.StreamHandler()
    streamHandler.setFormatter(formatter)

    l.setLevel(level)
    l.addHandler(fileHandler)
    l.addHandler(streamHandler)    


setup_logger('log1', txtName+"txt")
setup_logger('log2', txtName+"small.txt")
logger_1 = logging.getLogger('log1')
logger_2 = logging.getLogger('log2')




logger_1.info('111messasage 1')
logger_2.info('222ersaror foo')

Greatcoat

Are logger_1 and logger_2 global? I.e., I can use them inside any function? if not, is it a good idea to make them global in the def main function? if so, how would you go about it? – Kirovograd

you would not do that in the def, you would do that wherever you define your loggers. – Scheel

Wouldn't this cause duplicate console outputs? – Herringbone

@Herringbone If you log to console, yes. You can log to only an input file by excluding l.addHandler(streamHandler) – Nummary