How to write to a file, using the logging Python module?

Asked 17/6, 2011 at 13:44 Answered 8/7, 2022 at 3:27

319

How can I use the logging module in Python to write to a file? Every time I try to use it, it just prints out the message.

Gilpin answered 17/6, 2011 at 13:44 Comment(0)

402

An example of using logging.basicConfig rather than logging.fileHandler()

logging.basicConfig(filename=logname,
                    filemode='a',
                    format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s',
                    datefmt='%H:%M:%S',
                    level=logging.DEBUG)

logging.info("Running Urban Planning")

logger = logging.getLogger('urbanGUI')

In order, the five parts do the following:

set the output file (filename=logname)
set it to append rather than overwrite (filemode='a')
determine the format of the output message (format=...)
determine the format of the output time (datefmt='%H:%M:%S')
and determine the minimum message level it will accept (level=logging.DEBUG).

Luhey answered 17/6, 2011 at 14:3 Comment(2)

What's the purpose of he last line? i.e. logger = logging.getLogger('urbanGUI') – Burt 17/6, 2022 at 14:47

@Vichoko, it's not relevant for this answer, it's just an example of how to use the logger. See the documentation to learn what it does. – Ululate 17/6, 2022 at 15:12

133

Taken from the "logging cookbook":

# create logger with 'spam_application'
logger = logging.getLogger('spam_application')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)

And you're good to go.

P.S. Make sure to read the logging HOWTO as well.

Put answered 17/6, 2011 at 13:48 Comment(5)

To answer your first question, feel free to look at the title of the question I asked. I have gone over the link you provided and it was helpful. I have copied the code you gave me and was I wrong to assume I would be able to use logger.info("message") and logger.warning("message") successfully? I was able to write to the file using logger.warning, however logger.info does not seem to write to the file. – Gilpin 17/6, 2011 at 14:25

Try removing the setLevel call. Reading the handler docs it looks like all messages are processed by default. – Luhey 17/6, 2011 at 14:30

I can write out to file only using logger.warning("message"), cannot use logger.info("message") nor logger.debug("message"). That's a bit annoying. – Victoriavictorian 4/12, 2015 at 3:14

The code example @EliBendersky has written is missing 1 step if you want to write info / debug msgs. The logger itself needs its own log level to be configured to accept that level of logging messages e.g. logger.setLevel(logging.DEBUG). Loggers can be configured with multiple handlers; the level configured in the logger determines which severity level log messages to send to each of its handlers, & the levels set in the handlers determine which levels the handler will process. Note that those wanting to print info messages only need to set this to INFO in both the logger and the handler. – Selle 23/10, 2016 at 21:56

I've updated the sample to do logger.setLevel(logging.DEBUG) -- thanks for the comments – Put 13/12, 2017 at 18:50

Here are two examples, one prints the logs (stdout) the other writes the logs to a file:

import logging
import sys

logger = logging.getLogger()
logger.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s | %(levelname)s | %(message)s')

stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(logging.DEBUG)
stdout_handler.setFormatter(formatter)

file_handler = logging.FileHandler('logs.log')
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)


logger.addHandler(file_handler)
logger.addHandler(stdout_handler)

With this example, all logs will be printed and also be written to a file named logs.log

Usage example:

logger.info('This is a log message!')
logger.error('This is an error message.')

List of all built-in logging handlers https://docs.python.org/3/library/logging.handlers.html

Selfinsurance answered 23/11, 2020 at 14:14 Comment(0)

I prefer to use a configuration file. It allows me to switch logging levels, locations, etc without changing code when I go from development to release. I simply package a different config file with the same name, and with the same defined loggers.

import logging.config
if __name__ == '__main__':
    # Configure the logger
    # loggerConfigFileName: The name and path of your configuration file
    logging.config.fileConfig(path.normpath(loggerConfigFileName))

    # Create the logger
    # Admin_Client: The name of a logger defined in the config file
    mylogger = logging.getLogger('Admin_Client')

    msg='Bite Me'
    myLogger.debug(msg)
    myLogger.info(msg)
    myLogger.warn(msg)
    myLogger.error(msg)
    myLogger.critical(msg)

    # Shut down the logger
    logging.shutdown()

Here is my code for the log config file

#These are the loggers that are available from the code
#Each logger requires a handler, but can have more than one
[loggers]
keys=root,Admin_Client


#Each handler requires a single formatter
[handlers]
keys=fileHandler, consoleHandler


[formatters]
keys=logFormatter, consoleFormatter


[logger_root]
level=DEBUG
handlers=fileHandler


[logger_Admin_Client]
level=DEBUG
handlers=fileHandler, consoleHandler
qualname=Admin_Client
#propagate=0 Does not pass messages to ancestor loggers(root)
propagate=0


# Do not use a console logger when running scripts from a bat file without a console
# because it hangs!
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)# The comma is correct, because the parser is looking for args


[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=logFormatter
# This causes a new file to be created for each script
# Change time.strftime("%Y%m%d%H%M%S") to time.strftime("%Y%m%d")
# And only one log per day will be created. All messages will be amended to it.
args=("D:\\Logs\\PyLogs\\" + time.strftime("%Y%m%d%H%M%S")+'.log', 'a')


[formatter_logFormatter]
#name is the name of the logger root or Admin_Client
#levelname is the log message level debug, warn, ect 
#lineno is the line number from where the call to log is made
#04d is simple formatting to ensure there are four numeric places with leading zeros
#4s would work as well, but would simply pad the string with leading spaces, right justify
#-4s would work as well, but would simply pad the string with trailing spaces, left justify
#filename is the file name from where the call to log is made
#funcName is the method name from where the call to log is made
#format=%(asctime)s | %(lineno)d | %(message)s
#format=%(asctime)s | %(name)s | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno) | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno)04d | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno)4s | %(levelname)-8s | %(message)s

format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s


#Use a separate formatter for the console if you want
[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%(lineno)04d | %(message)s

Hyperthermia answered 10/6, 2015 at 9:44 Comment(2)

Naming the file with a date requires double %% in Python 3. e.g. time.strftime("%%Y%%m%%D") – Cadel 3/8, 2018 at 13:24

I've been on projects that require regression testing of all code affected by any change to a file, configuration files are the exception. It sounds draconian, but we had very high acceptance rates. – Hunchbacked 30/3, 2023 at 10:37

http://docs.python.org/library/logging.html#logging.basicConfig

logging.basicConfig(filename='/path/to/your/log', level=....)

Antepenult answered 17/6, 2011 at 13:48 Comment(2)

This saves the logs in the file, that's good. What if together with this, I would like it to log the outputs on the terminal too? – Schizogenesis 4/7, 2018 at 8:42

The official logging module documentation allows for this. You can even choose what logs go into the terminal and which ones go into a file, and many more interesting applications. docs.python.org/3/howto/… – Yodel 19/12, 2018 at 2:55

here's a simpler way to go about it. this solution doesn't use a config dictionary and uses a rotation file handler, like so:

import logging
from logging.handlers import RotatingFileHandler
     
logging.basicConfig(handlers=[RotatingFileHandler(filename=logpath+filename,
                     mode='w', maxBytes=512000, backupCount=4)], level=debug_level,
                     format='%(levelname)s %(asctime)s %(message)s', 
                    datefmt='%m/%d/%Y%I:%M:%S %p')
     
logger = logging.getLogger('my_logger')

or like so:

import logging
from logging.handlers import RotatingFileHandler
     
handlers = [ RotatingFileHandler(filename=logpath+filename, 
            mode='w', 
            maxBytes=512000, 
            backupCount=4)
           ]
logging.basicConfig(handlers=handlers, 
                    level=debug_level, 
                    format='%(levelname)s %(asctime)s %(message)s', 
                    datefmt='%m/%d/%Y%I:%M:%S %p')
     
logger = logging.getLogger('my_logger')

the handlers variable needs to be an iterable. logpath+filename and debug_level are just variables holding the respective info. of course, the values for the function params are up to you.

the first time i was using the logging module i made the mistake of writing the following, which generates an OS file lock error (the above is the solution to that):

import logging
from logging.handlers import RotatingFileHandler
     
logging.basicConfig(filename=logpath+filename, 
       level=debug_level, 
       format='%(levelname)s %(asctime)s %(message)s', 
       datefmt='%m/%d/%Y%I:%M:%S %p')
     
logger = logging.getLogger('my_logger')
logger.addHandler(RotatingFileHandler(
       filename=logpath+filename, 
       mode='w', 
       maxBytes=512000, 
       backupCount=4))

Lutetium answered 17/2, 2020 at 13:13 Comment(1)

It would be helpful to point out specifically what caused the OS file lock error, and I question the value of posting an example that fails without appropriate embedded comments for those who like to cut-n-paste! – Hunchbacked 30/3, 2023 at 10:41

http://docs.python.org/library/logging.handlers.html#filehandler

The FileHandler class, located in the core logging package, sends logging output to a disk file.

Psychoactive answered 17/6, 2011 at 13:45 Comment(2)

+1 For a complete example, see the "basic tutorial": docs.python.org/howto/logging.html#logging-to-a-file – Ferrol 17/6, 2011 at 13:50

I like how there are several different types of FileHandlers for various situations, too. (WatchedFileHandler, RotatingFileHandler, etc.) – Psychoactive 17/6, 2011 at 13:51

This example should work fine. I have added streamhandler for console. Console log and file handler data should be similar.

    # MUTHUKUMAR_TIME_DATE.py #>>>>>>>> file name(module)

    import sys
    import logging
    import logging.config
    # ================== Logger ================================
    def Logger(file_name):
        formatter = logging.Formatter(fmt='%(asctime)s %(module)s,line: %(lineno)d %(levelname)8s | %(message)s',
                                      datefmt='%Y/%m/%d %H:%M:%S') # %I:%M:%S %p AM|PM format
        logging.basicConfig(filename = '%s.log' %(file_name),format= '%(asctime)s %(module)s,line: %(lineno)d %(levelname)8s | %(message)s',
                                      datefmt='%Y/%m/%d %H:%M:%S', filemode = 'w', level = logging.INFO)
        log_obj = logging.getLogger()
        log_obj.setLevel(logging.DEBUG)
        # log_obj = logging.getLogger().addHandler(logging.StreamHandler())

        # console printer
        screen_handler = logging.StreamHandler(stream=sys.stdout) #stream=sys.stdout is similar to normal print
        screen_handler.setFormatter(formatter)
        logging.getLogger().addHandler(screen_handler)

        log_obj.info("Logger object created successfully..")
        return log_obj
    # =======================================================


MUTHUKUMAR_LOGGING_CHECK.py #>>>>>>>>>>> file name
# calling **Logger** function
file_name = 'muthu'
log_obj =Logger(file_name)
log_obj.info("yes   hfghghg ghgfh".format())
log_obj.critical("CRIC".format())
log_obj.error("ERR".format())
log_obj.warning("WARN".format())
log_obj.debug("debug".format())
log_obj.info("qwerty".format())
log_obj.info("asdfghjkl".format())
log_obj.info("zxcvbnm".format())
# closing file
log_obj.handlers.clear()

OUTPUT:
2019/07/13 23:54:40 MUTHUKUMAR_TIME_DATE,line: 17     INFO | Logger object created successfully..
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 8     INFO | yes   hfghghg ghgfh
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 9 CRITICAL | CRIC
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 10    ERROR | ERR
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 11  WARNING | WARN
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 12    DEBUG | debug
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 13     INFO | qwerty
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 14     INFO | asdfghjkl
2019/07/13 23:54:40 MUTHUKUMAR_LOGGING_CHECK,line: 15     INFO | zxcvbnm

Thanks,

Clobber answered 13/7, 2019 at 18:36 Comment(0)

import logging

from datetime import datetime

filename = datetime.now().strftime("%d-%m-%Y %H-%M-%S")#Setting the filename from current date and time
logging.basicConfig(filename=filename, filemode='a',
                    format="%(asctime)s, %(msecs)d %(name)s %(levelname)s [ %(filename)s-%(module)s-%(lineno)d ]  : %(message)s",
                    datefmt="%H:%M:%S",
                    level=logging.DEBUG)

asctime

%(asctime)s

Human-readable time when the LogRecord was created. By default this is of the form ‘2003-07-08 16:49:45,896’ (the numbers after the comma are millisecond portion of the time).
created

%(created)f

Time when the LogRecord was created (as returned by time.time()).
exc_info

You shouldn’t need to format this yourself.

Exception tuple (à la sys.exc_info) or, if no exception has occurred, None.
filename

%(filename)s

Filename portion of pathname.
funcName

%(funcName)s

Name of function containing the logging call.
levelname

%(levelname)s

Text logging level for the message ('DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL').
levelno

%(levelno)s

Numeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL).
lineno

%(lineno)d

Source line number where the logging call was issued (if available).
message

%(message)s

The logged message, computed as msg % args. This is set when Formatter.format() is invoked.
module

%(module)s

Module (name portion of filename).
msecs

%(msecs)d

Millisecond portion of the time when the LogRecord was created.
msg

You shouldn’t need to format this yourself.

The format string passed in the original logging call. Merged with args to produce message, or an arbitrary object (see Using arbitrary objects as messages).
name

%(name)s

Name of the logger used to log the call.
pathname

%(pathname)s

Full pathname of the source file where the logging call was issued (if available).
process

%(process)d

Process ID (if available).
processName

%(processName)s

Process name (if available).
relativeCreated

%(relativeCreated)d

Time in milliseconds when the LogRecord was created, relative to the time the logging module was loaded.
stack_info

You shouldn’t need to format this yourself.

Stack frame information (where available) from the bottom of the stack in the current thread, up to and including the stack frame of the logging call which resulted in the creation of this record.
thread

%(thread)d

Thread ID (if available).
threadName

%(threadName)s

Thread name (if available).

Head over to official python3 page for more info regarding logging.

Golub answered 9/2, 2022 at 5:37 Comment(0)

Although it is an old question, for people reaching this question these days, you can also use dictConfig. For example, for a file with info level and above :

logging.config.dictConfig({
    'version': 1,
    'formatters': {
        'default': {
            'format': '[%(asctime)s] %(message)s',
        }
    },
    'handlers': {
        'info': {
            'level': logging.INFO,
            'class': 'logging.FileHandler',
            'filename': 'info.log',
        },
    },
    "root": {
        "level": logging.INFO,
        "handlers": ["info"]
    }
})

Or another example to be more specific, with rotating file and in a specific directory :

import logging
from pathlib import Path
from datetime import date

today = date.today()
folder = './log'
Path(folder).mkdir(parents=True, exist_ok=False) # Create folder if not exists
logging.config.dictConfig({
        ...
        'info': {
            'level': logging.INFO,
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'filename': f'{folder}/info-{today.month:02}-{today.year}.log',
            # Roll over on the first day of the weekday
            'when': 'W0',
            # Roll over at midnight
            'atTime': datetime.time(hour=0),
            # Number of files to keep.
            'backupCount': 8
        },
        ...

Poachy answered 10/5, 2022 at 8:45 Comment(2)

How do we import Path? – Couching 24/1 at 17:31

Path class is coming from pathlib. It's edited with correct imports. – Poachy 25/1 at 12:14

import sys
import logging

from util import reducer_logfile
logging.basicConfig(filename=reducer_logfile, format='%(message)s',
                    level=logging.INFO, filemode='w')

Bainite answered 5/9, 2016 at 9:4 Comment(0)

-2

best and clean method.

    ds = datetime.now().strftime("%Y%m%d_%H%M%S")
    try:
        # py39 有force参数指定可能强制除去之前的handler，这里使用兼容写法，0708
        logging.getLogger().removeHandler(logging.getLogger().handlers[0])
        logging.getLogger().removeHandler(logging.getLogger().handlers[0])
    except:
        pass
    logging.basicConfig(
        # force=
        level=logging.INFO,
        format="%(asctime)s [%(levelname)s] %(message)s",
        handlers=[
            logging.FileHandler('log_%s_%s_%s.log' % (ds, create_net, os.path.basename(dataset_path))),
            logging.StreamHandler(sys.stdout)
        ]
    )
    logging.info('start train')

Eskill answered 8/7, 2022 at 3:27 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

best and clean method.

Recommended topics

Hot tags