Converting ConfigParser values to python data types

Question

ConfigParser requires all sections, keys and values to be strings; no surprise. It has methods to convert the values to datatypes with getfloat, getint, getboolean. If you don't know the datatype, you can wrap the get() with an eval() to get have the string evaluated such as:

>>> from ConfigParser import SafeConfigParser
>>> cp = SafeConfigParser()
>>> cp.add_section('one')
>>> cp.set('one', 'key', '42')
>>> print cp.get('one', 'key')
'42'
>>> print eval(cp.get('one', 'key'))
42
>>> cp.set('one', 'key', 'None')
>>> print eval(cp.get('one', 'key'))
None
>>> 

Is there a better way? I assume there some grave security concerns with evaluating text from a file- which I acknowledge; I completely trust the file.

I thought I would use pickle for this, but I would really like to keep the config file human readable.

How would you do it?

Glossator

Answer 1

If you are using Python 2.6 or above you can use ast.literal_eval:

ast.literal_eval(node_or_string)
Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.

This can be used for safely evaluating strings containing Python expressions from untrusted sources without the need to parse the values oneself.

This will work like eval when the string is safe:

>>> literal_eval("{'key': 10}")
{'key': 10}

But it will fail if anything besides the types listed in the documentation appear:

>>> literal_eval("import os; os.system('rm -rf somepath')")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/ast.py", line 49, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib64/python2.6/ast.py", line 37, in parse
    return compile(expr, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    import os; os.system('rm -rf somepath')
         ^
SyntaxError: invalid syntax

Cyndycynera

i used your answer to create this: codereview.stackexchange.com/questions/2775/… – Glossator

Answer 2

For those that may be looking for another easier answer, instead of having to convert the data types yourself, you can use the localconfig module that does the conversion for you. The conversion is done by guessing the data type based on the value (I.e. 123 is an int, 123.4 is a float, true is a bool, and so on).

Here is an example following the OP's:

>>> from localconfig import config
>>> config.read('[one]\nkey = 42\nkey2 = None')
>>> config.one.key, type(config.one.key)
(42, <type 'int'>)
>>> config.one.key2, type(config.one.key2)
(None, <type 'NoneType'>)
>>> config.get('one', 'key'), config.get('one', 'key2')
(42, None)

It is a wrapper on top of ConfigParser, so it is fully compatible.

Check it out at https://pypi.python.org/pypi/localconfig

Talkative

Thanks, your packages was very helpful for me! – Diplomacy

Answer 3

Here is an another solution. You can generate a getany() method that will automatically detect and return the correct data type for most types including str, int, float, bool and None.

Note that the notation in the config file needs to be Python for this to work. e.g. True for boolean true and None for null values.

from ast import literal_eval
import configparser

parser = configparser.ConfigParser(converters={"any": lambda x: literal_eval(x)})
value = parser.getany("section", "key")
# ...

Muire

This doesn't work for simple string values, does it? key=1.0 is fine, but key=test isn't... – Underthecounter

yes. As mentioned it only works for values in Python notation. – Muire

Answer 4

If you are on 2.7+ then you can use the .getint .getfloat .getbool methods. You can learn more about them in the docs

So your application would use print cp.getint('one', 'key')

Hankhanke

Answer 5

Checkout ConfigIt for more pythonic configuration options

https://github.com/barberj/ConfigIt

Superorganic

Answer 6

Adding on to Erik Kalkoken's answer. You can probably write a converter function which gives you more control over how you want to evaluate the value.

def converter(in_str: str):
    try:
        out = literal_eval(in_str)
    except Exception:
        out = in_str
    return out

parser = configparser.ConfigParser(converters={"any": lambda x: converter(x)})
value = parser.getany("section", "key")

Humidistat