parsing .properties file in Python

Question

The ConfigParser module raises an exception if one parses a simple Java-style .properties file, whose content is key-value pairs (i..e without INI-style section headers). Is there some workaround?

Desai

It seems to me the Properties format (docs.oracle.com/javase/6/docs/api/java/util/…) is somewhat more expressive than ConfigParser style ini files. Hence it's probably not a good idea to try and shoehorn it. Instead try jython or this snippet: code.activestate.com/recipes/… – Colorful

Answer 1

Say you have, e.g.:

$ cat my.props
first: primo
second: secondo
third: terzo

i.e. would be a .config format except that it's missing a leading section name. Then, it easy to fake the section header:

import ConfigParser

class FakeSecHead(object):
    def __init__(self, fp):
        self.fp = fp
        self.sechead = '[asection]\n'

    def readline(self):
        if self.sechead:
            try: 
                return self.sechead
            finally: 
                self.sechead = None
        else: 
            return self.fp.readline()

usage:

cp = ConfigParser.SafeConfigParser()
cp.readfp(FakeSecHead(open('my.props')))
print cp.items('asection')

output:

[('second', 'secondo'), ('third', 'terzo'), ('first', 'primo')]

Encephalomyelitis

would be great if there was an option in configparser to suppress that exception, for the sake of mere mortals like me :) – Desai

great solution, but it can be shortened a lot: def FakeSecHead(fp): yield '[asection]\n'; yield from fp – Rosemonde

would that work for writing into the config file too? – Monochord

Answer 2

I thought MestreLion's "read_string" comment was nice and simple and deserved an example.

For Python 3.2+, you can implement the "dummy section" idea like this:

with open(CONFIG_PATH, 'r') as f:
    config_string = '[dummy_section]\n' + f.read()
config = configparser.ConfigParser()
config.read_string(config_string)

Nymphomania

Elegant. Except for the fact that you need to ensure that the CONFIG_PATH, a.k.a configuration file, exists. Which configparsers built-in does for you. But I guess that's just a try away ;-) – Osber

Answer 3

My solution is to use StringIO and prepend a simple dummy header:

import StringIO
import os
config = StringIO.StringIO()
config.write('[dummysection]\n')
config.write(open('myrealconfig.ini').read())
config.seek(0, os.SEEK_SET)

import ConfigParser
cp = ConfigParser.ConfigParser()
cp.readfp(config)
somevalue = cp.getint('dummysection', 'somevalue')

Zworykin

Added the needed \n and removed the unnecessary 'r' mode on the open() call. – Blancmange

I find this the simplest solution. – Desai

I have newlines in my ini, how to work with that? Ie: one setting has it's several entries, one on it's own line. – Create

This is a nice quick solution, but note that a Java properties file could use features that could break a ConfigParser, e.g., ! as comment or \ (backslash) for line continuation and escapes (among others). More details of such features can be found here: docs.oracle.com/javase/7/docs/api/java/util/Properties.html. – Clove

Answer 4

Alex Martelli's answer above does not work for Python 3.2+: readfp() has been replaced by read_file(), and it now takes an iterator instead of using the readline() method.

Here's a snippet that uses the same approach, but works in Python 3.2+.

>>> import configparser
>>> def add_section_header(properties_file, header_name):
...   # configparser.ConfigParser requires at least one section header in a properties file.
...   # Our properties file doesn't have one, so add a header to it on the fly.
...   yield '[{}]\n'.format(header_name)
...   for line in properties_file:
...     yield line
...
>>> file = open('my.props', encoding="utf_8")
>>> config = configparser.ConfigParser()
>>> config.read_file(add_section_header(file, 'asection'), source='my.props')
>>> config['asection']['first']
'primo'
>>> dict(config['asection'])
{'second': 'secondo', 'third': 'terzo', 'first': 'primo'}
>>>

Swirly

Python 3.2 also added read_string(), which makes appending the dummy section a trivial task. – Heptameter

The add_section_header can simply be: config.read_file(itertools.chain(['[SECTION_NAME]'], file)) – Braga

Answer 5

with open('some.properties') as file:
    props = dict(line.strip().split('=', 1) for line in file)

Credit to How to create a dictionary that contains key‐value pairs from a text file

maxsplit=1 is important if there are equal signs in the value (e.g. someUrl=https://some.site.com/endpoint?id=some-value&someotherkey=value)

Leguminous

This will error if the file has comments. We can avoid by using this: dict(line.strip().split('=', 1) for line in file if not line.startswith("#") and not len(line.strip()) == 0) – Zillion

Answer 6

YAY! another version

Based on this answer (the addition is using a dict, with statement, and supporting the % character)

import ConfigParser
import StringIO
import os

def read_properties_file(file_path):
    with open(file_path) as f:
        config = StringIO.StringIO()
        config.write('[dummy_section]\n')
        config.write(f.read().replace('%', '%%'))
        config.seek(0, os.SEEK_SET)

        cp = ConfigParser.SafeConfigParser()
        cp.readfp(config)

        return dict(cp.items('dummy_section'))

Usage

props = read_properties_file('/tmp/database.properties')

# It will raise if `name` is not in the properties file
name = props['name']

# And if you deal with optional settings, use:
connection_string = props.get('connection-string')
password = props.get('password')

print name, connection_string, password

---

the .properties file used in my example

name=mongo
connection-string=mongodb://...
password=my-password%1234

Edit 2015-11-06

Thanks to Neill Lima mentioning there was an issue with the % character.

The reason for that is ConfigParser designed to parse .ini files. The % character is a special syntax. in order to use the % character simply added a a replace for % with %% according to .ini syntax.

Eiffel

This solution worked well for me up to the point I had a password with '%' - no single quotes, and it crashed the ConfigParser. Python 2.7.6 – Celebrant

your solution worked for me but I had to adapt for multiple lines with an additional .replace('\\\n', '')) Basically any additional condition can go with the write... replace – Probation

.replace('%', '%%') gives incorrect results if there are multiple % in the source. It is better to use a dummy interpolation instead of .replace('%', '%%'): configparser.ConfigParser(interpolation=configparser.Interpolation()) which doesn't change % in any way. Python 3.9 – Palisade

Answer 7

from pyjavaproperties import Properties
p = Properties()
p.load(open('test.properties'))
p.list()
print p
print p.items()
print p['name3']
p['name3'] = 'changed = value'
print p['name3']
p['new key'] = 'new value'
p.store(open('test2.properties','w'))

Washedout

The only person to use an actual library made for Java properties files. I will use this and get back on its ease of use compared to others. – Yapon

Answer 8

This answer suggests using itertools.chain in Python 3.

from configparser import ConfigParser
from itertools import chain

parser = ConfigParser()
with open("foo.conf") as lines:
    lines = chain(("[dummysection]",), lines)  # This line does the trick.
    parser.read_file(lines)

Popovich

Answer 9

with open('mykeyvaluepairs.properties') as f:
    defaults = dict([line.split() for line in f])
config = configparser.ConfigParser(defaults)
config.add_section('dummy_section')

Now config.get('dummy_section', option) will return 'option' from the DEFAULT section.

or:

with open('mykeyvaluepairs.properties') as f:
    properties = dict([line.split() for line in f])
config = configparser.ConfigParser()
config.add_section('properties')
for prop, val in properties.items():
    config.set('properties', prop, val)

In which case config.get('properties', option) doesn't resort to the default section.

Anguished

Answer 10

Yet another answer for python2.7 based on Alex Martelli's answer

import ConfigParser

class PropertiesParser(object):

    """Parse a java like properties file

    Parser wrapping around ConfigParser allowing reading of java like
    properties file. Based on stackoverflow example:
    https://mcmap.net/q/152702/-parsing-properties-file-in-python/2819788#2819788

    Example usage
    -------------
    >>> pp = PropertiesParser()
    >>> props = pp.parse('/home/kola/configfiles/dev/application.properties')
    >>> print props

    """

    def __init__(self):
        self.secheadname = 'fakeSectionHead'
        self.sechead = '[' + self.secheadname + ']\n'

    def readline(self):
        if self.sechead:
            try:
                return self.sechead
            finally:
                self.sechead = None
        else:
            return self.fp.readline()

    def parse(self, filepath):
        self.fp = open(filepath)
        cp = ConfigParser.SafeConfigParser()
        cp.readfp(self)
        self.fp.close()
        return cp.items(self.secheadname)

Avian