Python logging file config KeyError: 'formatters'

Question

I'm currently working on a python project and I set up logging using a config file. It has already worked and was logging my messages as wanted.

But then, after rearranging some of the packages and modules, I only get a key error.

Full Traceback:

    Traceback (most recent call last):
  File "/Volumes/Daten/Eclipse/workspace/Carputer/src/pyboard/__init__.py", line 42, in <module>
    logging.config.fileConfig('../logging.conf', disable_existing_loggers=False)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/logging/config.py", line 70, in fileConfig
    formatters = _create_formatters(cp)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/logging/config.py", line 103, in _create_formatters
    flist = cp["formatters"]["keys"]
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py", line 937, in __getitem__
    raise KeyError(key)
KeyError: 'formatters'

Here is my logging file:

[loggers]
keys=root,pyBoard

[handlers]
keys=consoleHandler

[formatters]
keys=detailedFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_pyBoard]
level=DEBUG
handlers=consoleHandler
qualname=pyBoard
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=detailedFormatter
args=(sys.stdout,)

[formatter_detailedFormatter]
format=%(asctime)s - %(name)s - %(levelname)s : Line %(lineno)s - %(message)s
datefmt=

And the relevant code:

if __name__ == '__main__':

    logging.config.fileConfig('../logging.conf', disable_existing_loggers=False)
    logger = logging.getLogger(__name__)

    obc = Onboard_computer('/dev/ttys001')
    obc.run()

It is almost the same as from the Python Logging Tutorial. I really don't get why it is not working and it drives crazy. It worked, I changed nothing on the code nor on the setup, and it just stopped working and Python throws this KeyError.

My setup: Mac OS X 10.9.2, Eclipse Kepler with PyDev and Python 3.3. I also tested it on a Raspberry Pi with Raspbian Wheezy and Python 3.2 and in Eclipse with Python 2.7 (same error).

Does anyone of you guys have a clue?

Bova

The error suggests it is looking for a different file logging.config.fileConfig('../logging.ini', disable_existing_loggers=False) It is looking for one ending in .ini where as your code has one ending in .conf? – Espinosa

Typo, sorry... I just installed PyCharm and tested it there with the same installation of Python 3.3.5. And: it works! Any ideas? – Bova

Hi moritz, did you manage to resolve this issue? I'm currently having the same problem when trying to run my application with logging on the RPi. – Susysuter

Answer 1

I had this issue because Python couldn't find my config file, though you would never know it by the error message. Apparently it does not look for the config file relative to the file in which the code is running, but rather relative to the current working directory (which you can get from os.getcwd()). I used the following code to initialize the logger. The log.config file is in the same directory as the file running this code:

from os import path
log_file_path = path.join(path.dirname(path.abspath(__file__)), 'log.config')
logging.config.fileConfig(log_file_path)

Iorgo

Also check to make sure your logging config file really exists where you think it does. – Fianna

Yeah, the error message is useless. Thanks to both @Iorgo and @Fianna - what was missing in my case was include path/to/logging.ini in my manifest file, and hence the file wasn't included in my wheel package! And it worked on my local machine but died as soon as deployed to AWS Lambda! – Hanahanae

Thanks, that was it. Up voted your response :). I was able to specify a relative path and it worked. Just for the people who use VS Code with the Python extension from Microsoft: no matter where you class or script is, the default current working directory is the top folder of your workspace. – Illconsidered

This was beautiful, it was unable to find proper path no matter what i tried. This worked like a charm. – Aborn

Answer 2

d512 was correct. And when running the code and based on the PYTHONPATH, Python treats your project root directory as the 'current path' no matter where is your running file located. So another way is to add the relative path either as following:

import logging
logging.config.fileConfig('root_path_of_project/.../logging.conf')

or relative to the current file:

LOGGING_CONFIG = Path(__file__).parent / 'logging.conf'
...
logging.config.fileConfig(LOGGING_CONFIG)

Hope it helps

Dealer

Answer 3

In my case, I was missing the following from my logging.conf file -

[formatters]
keys = json

Below is the structure of the entire file -

[loggers]
keys = root,__main__

[logger_root]
handlers =

[logger___main__]
level = INFO
handlers = __main__
qualname = __main__

[handlers]
keys = __main__

[handler___main__]
class = StreamHandler
level = INFO
formatter = json
args = (sys.stdout,)

[formatters]
keys = json

[formatter_json]
format=%(asctime)f %(module)s %(levelname)s %(message)s %(funcName)s
class = pythonjsonlogger.jsonlogger.JsonFormatter

Python file that uses the logger -

import logging.config
from os import path

log_file_path = path.join(path.dirname(path.abspath('logging.conf')), 'logging.conf')

logging.config.fileConfig(log_file_path, disable_existing_loggers=False)
logger = logging.getLogger(__name__)

logger.info('random message')  
#{"asctime": "2021-10-22 02:48:26,532", "module": "jsonLogger", "levelname": "INFO", "message": "random message", "funcName": "<module>"}

Don't forget to install python-json-logger btw.

Galloway

Answer 4

Try to replace

logging.config.fileConfig('../logging.conf', disable_existing_loggers=False)

with this

logging.config.fileConfig('logging.conf', disable_existing_loggers=False)

Not sure, but probably your logging.conf is in your current working directory, and with the .. the file cannot be found.

Overanxious

Answer 5

Some of my logging settings were in migrations/alembic.ini.

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic

Instead of messing with it, I deleted the migrations folder, deleted the alembic_version table in my db, and then ran flask db stamp heads and then could do my normal db actions when I add a column again.

# flask db init         # only needed when you don't have any db setup
flask db stamp head
flask db migrate
flask db upgrade

Helot

Answer 6

In addition to the most voted version a note if you get this error only when installing your package from Pypi:

you need to add your logging file to setup.py package_data field like so:

setup(
    name="...",
    #...
    packages=["mypkg"],
    package_data={
        "mypkg": [
            "logging.conf", # <- otherwise it will not be included in your package
        ]
    },

Membranophone

Answer 7

in my case this error was showing up when I was trying to run my flask app on Docker container.

What helped me is adding:

flask db init

into the boot.sh that runs at the beginning of container creation.

Gradate