how to set logging level from command line

Question

I'm using argparse to get the logging level from the command line and then passing it as input for logging.basicConfig. However, the way I'm trying to implement this is not working. Any suggestion?

Desire behavior, from command line:

python main.py -log=DEBUG

Desire output

DEBUG:__main__: Debug is working

Code

import logging
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("-log", "--log", nargs='+', help="Provide logging level. Example --log debug'")

log_level = parser.parse_args().log
log_level = 'logging.'+log_level[0]
print(log_level)
logging.basicConfig(level=log_level)
logger = logging.getLogger(__name__) 
logger.debug(' Debug is working')

Polygynous

Why do you put nargs='+', the user have to choose only one level, the default notation is that your parameter is optional (python docs)? – Calicut

@Calicut you are right, nargs='+' is not required and was the main reason why I was getting a list as input. – Polygynous

I often recommend args = parser.parse_args() followed by print(args). The gives you a clear(er) idea of what the parser has done for you. – Bireme

Also see stackoverflow.com/questions/14097061 – Zone

@Bireme Thanks a lot! That was helpful! You inspired me and now my args = parser.parse_args() is followed by for arg, val in vars(args).items(): print(f"{arg:17s} : '{val}'"). This gives me an even clearer idea of what the parser has done for me ! – Abbey

Answer 1

The basicConfig function can take a string argument for level, and it will check its validity for you, so the code can be much simpler than BarryPye's answer.

import argparse
import logging

parser = argparse.ArgumentParser()
parser.add_argument( '-log',
                     '--loglevel',
                     default='warning',
                     help='Provide logging level. Example --loglevel debug, default=warning' )

args = parser.parse_args()

logging.basicConfig( level=args.loglevel.upper() )
logging.info( 'Logging now setup.' )

Wilke

This would work better (more readable than replicating the logging levels in a local dict), but I would recommend limiting the possible inputs in this case, with choices=logging._nameToLevel.keys() – Samuelsamuela

Correct me if I am wrong, and I know this wasn't explicitly stated in the question, but while basicConfig works fine for the main script, it does nothing to modify logger levels in imported packages, which I assume most non-trivial uses of logging would want to do. I tried just this, hardcoded logging.INFO once in basic.Config. Main got INFO, but package stayed at WARNING. Changed to ERROR and main was at ERROR. package stayed at WARNING. – Dichogamy

Even if basicConfig did affect logging levels on packages, the problem is that the logger = logging.get_logger(__name__) on the packages will fire as you import those packages, which is before you hit argparse/optparse parsing, provided you respect PEP 8 to import packages before running code. I don't mean to criticize this answer, which I am upvoting, but logging seems to always lose me in the weeds. – Dichogamy

Answer 2

Putting these combinations together, allowing the user to name the level with upper or lower case, allowing only one level to be specified, and picking the explicit level from a dictionary with a default to the WARNING level:

import argparse
import logging
parser = argparse.ArgumentParser()
parser.add_argument(
    "-log", 
    "--log", 
    default="warning",
    help=(
        "Provide logging level. "
        "Example --log debug', default='warning'"),
)

options = parser.parse_args()
levels = {
    'critical': logging.CRITICAL,
    'error': logging.ERROR,
    'warn': logging.WARNING,
    'warning': logging.WARNING,
    'info': logging.INFO,
    'debug': logging.DEBUG
}
level = levels.get(options.log.lower())
if level is None:
    raise ValueError(
        f"log level given: {options.log}"
        f" -- must be one of: {' | '.join(levels.keys())}")
logging.basicConfig(level=level)
logger = logging.getLogger(__name__)

Aneroidograph

You can check the user enters a proper log level using the ArgumentParser.add_argument() choices input parameter. It allows you to provide a list of the possible inputs for that command line argument, and argparse handles the errors when the user enters the wrong argument. – Sextuple

About levels, you can use logging.getLevelName(options.log.upper()) and check if result is int. From py3.11 you can use also logging.getLevelNamesMapping() – Gatehouse

it cannot be this complicated!!! Oh my god, were I more competent, I would change the logging world. – Cicely

Answer 3

The level should be a variable from logging not a string, logging.DEBUG for example. I think you have to create a dict matching the given argument and the logging variable:

level_config = {'debug': logging.DEBUG, 'info': logging.INFO} # etc.
log_level = level_config[parser.parse_args().log[0].lower()]

You can also add choices=['debug', 'info', 'warning'] in your add_argument call.

Calicut

Hi, I tried your suggestion and got the following error: AttributeError: 'list' object has no attribute 'lower' – Polygynous

I forgot a [0] after the log (I edit my post). But I think blues solution is more elegant than mine. – Calicut

Nicely done. This one should be selected as answer! – Aneurysm

As of Python 3.2 you can specify level as a string – Guzman

Answer 4

log_level = 'logging.'+log_level[0] this just makes the string 'logging.DEBUG' which is not something that basicConfig understands. what it want's is the logging.DEBUG constant which you can get by getattr(logging, log_level[0]). Newer versions of python actually accept the textual representation as well and you can just pass in 'DEBUG' as level.

Temperance

Hi @blues, after trying log_level = getattr(logging, log_level[0]), logging.basicConfig(level=log_level) I got the following error raise TypeError("Level not an integer or a valid string: %r" % level) TypeError: Level not an integer or a valid string: <function info at 0x7ffff6cf0378> – Polygynous

My guess would be that you did not capitalize the argument. Make sure to supply the argument as -log=DEBUG and not -log=debug. Or call .upper() on it. – Temperance

Answer 5

Consolidating from various comments on other answers, from Python 3.11 a version that restricts your input to the available log levels looks like:

import argparse
import logging

parser = argparse.ArgumentParser()
parser.add_argument("-l", "--loglevel", default=logging.WARN,
       choices=logging.getLevelNamesMapping().keys(), help="Set log level")

args = parser.parse_args()
logging.basicConfig(level=args.loglevel)
logger = logging.getLogger(__name__)

Help text from above:

usage: log.py [-h] [-l {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}]

options:
  -h, --help            show this help message and exit
  -l {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}, --loglevel {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}
                        Set log level

Error if you use an invalid value for -l:

usage: log.py [-h] [-l {CRITICAL,FATAL,ERROR,WARN,WARNING,INFO,DEBUG,NOTSET}]
log.py: error: argument -l/--loglevel: invalid choice: 'BAD' (choose from 'CRITICAL', 'FATAL', 'ERROR', 'WARN', 'WARNING', 'INFO', 'DEBUG', 'NOTSET')

Grandioso