class RotatingFileHandler( filename[, mode[, maxBytes[, backupCount]]])
Returns a new instance of the RotatingFileHandler class. The specified file is opened and used as the stream for logging. If mode is not specified, a is used. By default, the file grows indefinitely.
A RotatingFileHandler allows us to rotate our log statements into a new file every time the current log file reaches a certain size.
In this example we’ll set it up so that when it reaches 500 bytes we’ll rotate into a new file up to a maximum number of 2 backups.
import logging
import logging.handlers as handlers
import time
logger = logging.getLogger('my_app')
logger.setLevel(logging.INFO)
logHandler = handlers.RotatingFileHandler('app.log', maxBytes=500, backupCount=2)
logHandler.setLevel(logging.INFO)
logger.addHandler(logHandler)
def main():
while True:
time.sleep(1)
logger.info("A Sample Log Statement")
main()
Upon execution of this you should notice that every time app.log exceeds 500 bytes, it is then closed and renamed app.log.x where the value of x increments till it reaches whatever we have set backupCount to.
class TimedRotatingFileHandler( filename [,when [,interval [,backupCount]]])
Returns a new instance of the TimedRotatingFileHandler class. The specified file is opened and used as the stream for logging. On rotating it also sets the filename suffix. Rotating happens based on the product of when and interval.
You can use the when to specify the type of interval. The list of possible values is, note that they are not case sensitive:
| Value | Type of interval |
|:--------:|:---------------------:|
| S | Seconds |
| M | Minutes |
| H | Hours |
| D | Days |
| W | Week day (0=Monday) |
| midnight | Roll over at midnight |
TimedRotatingFileHandler allows us to capture log files by a time slice.
import logging
import logging.handlers as handlers
import time
logger = logging.getLogger('my_app')
logger.setLevel(logging.INFO)
logHandler = handlers.TimedRotatingFileHandler('timed_app.log', when='M', interval=1)
logHandler.setLevel(logging.INFO)
logger.addHandler(logHandler)
def main():
while True:
time.sleep(1)
logger.info("A Sample Log Statement")
main()
Running this code will then create new log files every minute indefinitely. We can set the backupCount parameter on our logHandler instance and it will cap the number of log files we create.
Use of Appropriate Log Levels
With the TimedRotatingFileHandler and the RotatingFileHandler it is possible to do the following things such as log all error messages to a rotating file, but all normal log files to a TimedRotatingFileHandler as we hope that we can expect far more of them than error messages.
Two levels of records are split out two different log levels: INFO and ERROR to two distinct places.
import logging
import logging.handlers as handlers
import time
logger = logging.getLogger('my_app')
logger.setLevel(logging.INFO)
## Here we define our formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logHandler = handlers.TimedRotatingFileHandler('normal.log', when='M', interval=1, backupCount=0)
logHandler.setLevel(logging.INFO)
logHandler.setFormatter(formatter)
errorLogHandler = handlers.RotatingFileHandler('error.log', maxBytes=5000, backupCount=0)
errorLogHandler.setLevel(logging.ERROR)
errorLogHandler.setFormatter(formatter)
logger.addHandler(logHandler)
logger.addHandler(errorLogHandler)
def main():
while True:
time.sleep(1)
logger.info("A Sample Log Statement")
logger.error("An error log statement")
main()
You should notice that 3 log files are created when you run this. The error.log will contain only logs that are of level ERROR or higher. The normal.log will contain a combination of all log messages logged out of our application.
These are just a few things I feel are important when implementing your own logging system.
