How can I log a dictionary into a log file?

Question

I have a dictionary:

 d = {name : "John", age: 10}. 

And a log file setup as:

logging.basicConfig(level = logging.DEBUG, filename = "sample.log")

Is it possible to log this dictionary into the "sample.log" file? If yes, how can I do it?

Rosen

I have to log into the same file...as there are other logs also being logged. – Rosen

Answer 1

Simple solution that works

The logging functions will convert the '%s' to string representation (and if the object happens to be a container, then repr() will be used for the contained objects)

import logging
logging.basicConfig(level=logging.DEBUG, filename='sample.log')
logging.debug('Sample dict log: %s', {'name' : "John", 'age': 10})

How it shows in the log file:

DEBUG:root:Sample dict log: {'age': 10, 'name': 'John'}

If you have custom objects (e.g. instances of your own classes), you should implement a sensible __str__ and/or __repr__ for it, and the above will work without any hacks.

More on this here What is the difference between __str__ and __repr__?

A Note on performance

Notice that

logging.debug('%s', some_dict) is not same as

logging.debug('{0}'.format(some_dict))

In the first one, the function is passed 2 arguments '%s' and the original some_dict.

In the second one, the function is passed one argument which is the already-converted some_dict to a string. Because a function needs the arguments evaluated, the second example will always do the conversion, even if logging configs have turned off the logging of debug messages.

That's an unnecessary performance penalty. In the the first example, the logging.debug function can decide to do the conversion, or not.

JSON is not very good for this

For objects that aren't JSON-like (object instances, datetime, etc), json.dumps() would require a custom serializer e.g. with datetime objects you get:

TypeError: datetime.datetime(...) is not JSON serializable

This goes for any type that isn't supported by json.dumps()

Whereas the logging module would figure out a good representation for the object

Favian

You can coerce types which are not handled by json.dumps() using json hooks as in here: #8793948 – Englert

On Raspberry Pi this only worked when the dictionary did not contain certain information. Seems the pprint solution universal. – Sisak

Answer 2

Simple you can use

dict_data = {"test":"data"}
logger.info("Loging dict ---> {0}".format(dict_data))

Disfavor

I wouldn't do this. It's bad practice to format things in loggers because the format will always be evaluated. If your dict is massive and you have logLevel set to ERROR, the dict will still be converted to a string despite not being printed anywhere. – Colleen

Answer 3

you can convert it to string:

string_dictionary = str(d)

and then log the string dictionary variable to the file

using JSON

import json

d = {"name":"John","age":10}
json_string = json.dumps(d)

and if you want to convert the string back:

d = json.loads(json_string)

Virnelli

is it a good idea to convert it into a string and then store it ? – Rosen

the logging file is a text file so whatever the method you will follow to log it, it will ends with converting the dictionary to string and the second way is to convert it to json if you want to use it again for json it will be json.dumps(d) offcourse after importing json – Virnelli

yea...i was looking for json.dumps Can you show a sample for the code above ?? – Rosen

sure, import json d = {"name":"John","age":10} json_string = json.dumps(d) and if you want to convert the string back: d = json.loads(json_string) – Virnelli

so then i can directly put json_string into log files?? – Rosen

Answer 4

The problem with str(dictionary) and json.dumps(dictionary) is that the output can human unfriendly, especially if the dictionary is big and has nested structures.

If that's the case, you can Python's built-in pprint to pretty format the dictionary before logging:

import pprint

my_complex_dict = pprint.pformat(my_complex_dict)
logger.info(f"My complex dict:\n{my_complex_dict}")

Zebadiah

Not sure if there was this option back in 2016, but now you can simply add indent parameter to json.dumps and you'll get nicely formated dictionary. Check documentation – Notch

Answer 5

I came to this question when I wanted to log JSON lines for Cloudwatch.

I ended up using python-json-logger.

Install it: pip install python-json-logger

Use pythonjsonlogger.jsonlogger.JsonFormatter as the formatter class.

Wouldbe