Update INI file without removing comments

Question

Consider the following INI file:

[TestSettings]
# First comment goes here
environment = test

[Browser]
# Second comment goes here
browser = chrome
chromedriver = default

...

I'm using Python 2.7 to update the ini file:

config = ConfigParser.ConfigParser()
config.read(path_to_ini)
config.set('TestSettings','environment',r'some_other_value')

with open(path_to_ini, 'wb') as configfile:
    config.write(configfile)

How can I update the INI file without removing the comments. The INI file is updated but the comments are removed.

[TestSettings]
environment = some_other_value

[Browser]
browser = chrome
chromedriver = default

Besought

You cannot do it with ConfigParser. You need to use some other library. – Aswarm

have you tried the allow_no_value parameter? #6621137 – Barrows

allow_no_value has no affect over reading configuration. That is, comments are not read in the first place to get written down... – Impanel

did you find out how to do this? May be you could add your solution ? – Grandmotherly

Had to switch from INI file to XML. – Besought

Answer 1

The reason that comments in config files are wiped when writing back is that the write method didn't take care of comments at all. It just writes key/value pairs.

The easiest way to bypass this is to init configparser object with a customized comment prefix and allow_no_value = True. Then, if we want to keep the default "#" and ";" as comment lines in the file, we can specify another comment prefix, like "/" with comment_prefixes='/'. You can read this section of the configparser documentation for further information.

i.e., to keep comments, you have to trick configparser into believing that lines starting with "#" are not comments, but they are keys without a value. Interesting :)

# set comment_prefixes to a string which you will not use in the config file
config = configparser.ConfigParser(comment_prefixes='/', allow_no_value=True)
config.read_file(open('example.ini'))
...
config.write(open('example.ini', 'w'))

Watkins

Unofrtunately, this doesn't work for me: cp = ConfigParser.ConfigParser(allow_no_value=True, comment_prefixes='/') TypeError: __init__() got an unexpected keyword argument 'comment_prefixes' maybe this works only with newer versions of Configparser – Bloater

Ok so i just tried it out with python3, there it works. With python2 the error message above will be printed – Bloater

This doesn't help with commented lines, they're still deleted – Aviate

This ended up making all my comments lower case – Skep

@Skep This is because there is a default converter to configparser. You can bypass it by doing config.optionxform = lambda option: option. See this link – Nares

Nice workaround. For me it first still wiped certain comments cause I used bulk editing of sections: config["SECTION"] = {"key1": "value1", "key2": "value2"}. So keep in mind to always just set key value pairs individually. – Elector

This doesn't work very well at all - it fails for duplicate comments, and comments at the top of the file. – Dishman

Answer 2

ConfigObj preserves comments when reading and writing INI files, and seems to do what you want. Example usage for the scenario you describe :

from configobj import ConfigObj

config = ConfigObj(path_to_ini)
config['TestSettings']['environment'] = 'some_other_value'
config.write()

Oehsen

ConfigObj is obsolete. See a fork github.com/DiffSK/configobj – Matt

Answer 3

ConfigUpdater can update .ini files and preserve comments: pyscaffold/configupdater.

I don't know if it works for Python 2 though.

From the docs:

The key differences to ConfigParser are:

• minimal invasive changes in the update configuration file,
• proper handling of comments,

Constantine

Answer 4

ConfigObj is the best option in almost all cases.

Nevertheless, it does not support multiline values without triple quotes, like ConfigParser do. In this case, a viable option can be iniparse.

For example:

[TestSettings]
# First comment goes here
multiline_option = [
        first line,
        second line,
    ]

You can update the multiline value in this way.

import iniparse
import sys

c = iniparse.ConfigParser()
c.read('config.ini')
value = """[
    still the first line,
    still the second line,
]
"""
c.set('TestSettings', 'multiline_option', value=value)
c.write(sys.stdout)

Stich

Answer 5

Unless the configparser changes their implementation, all items not in option and section will not be read, so that when you write it back, un-read item is lost. You may write your update as follows:

def update_config(file, section, option, value, comment: str = None):
    sectFound = False
    lineIdx = 0
    with open(file, 'r') as config:
        lines = config.readlines()
        lineCount = len(lines)
        for line in lines:
            lineIdx += 1
            if sectFound and line.startswith('['):  #next secion
                lineIdx += -1
                lines.insert(lineIdx, option + ' = ' + value)
                if comment is not None:
                    lineIdx += 1
                    lines.insert(lineIdx, option + ' = ' + comment)
                break
            elif sectFound and line.startswith(option + ' = '):
                lines.pop(lineIdx)
                lines.insert(lineIdx, option + ' = ' + value)
                if comment is not None:
                    lineIdx += 1
                    lines.insert(lineIdx, option + ' = ' + comment)
                break
            elif sectFound and lineIdx == lineCount:
                lineIdx += 1
                lines.insert(lineIdx, option + ' = ' + value + '\n')
                if comment is not None:
                    lineIdx += 1
                    lines.insert(lineIdx, comment + '\n')
                break
            if line.strip() == '[' + section + ']':
                sectFound = True
    with open(file, 'w') as cfgfile:
        cfgfile.writelines(lines)
        if sectFound == False:
            cfgfile.writelines('[' + section + ']\n' + option + ' = ' + value)
            if comment is not None:
                cfgfile.writelines(comment)

Matland

Answer 6

If you pre-process your config file before parsing with ConfigParser, and post-process the output written by the ConfigParser, then you can update an INI file without removing comments.

I suggest to transform each comment into an option (key/value pair) during pre-processing. Then ConfigParser will then not throw the comments out. During post processing you'd then 'unpack' the comment and restore it.

To simplify the process you may want to subclass the ConfigParser and override the _read and write methods.

I've done this and posted the CommentConfigParser class in this Gist. It has one limitation. It does not support indented section headers, comments and keys. They should have no leading whitespace.

class CommentConfigParser(configparser.ConfigParser):
"""Comment preserving ConfigParser.

Limitation: No support for indenting section headers,
comments and keys. They should have no leading whitespace.
"""

def __init__(self, *args, **kwargs):
    super().__init__(*args, **kwargs)
    # Backup _comment_prefixes
    self._comment_prefixes_backup = self._comment_prefixes
    # Unset _comment_prefixes so comments won't be skipped
    self._comment_prefixes = ()
    # Template to store comments as key value pair
    self._comment_template = "#{0} = {1}"
    # Regex to match the comment id prefix
    self._comment_regex = re.compile(r"^#\d+\s*=\s*")
    # List to store comments above the first section
    self._top_comments = []

def _read(self, fp, fpname):
    lines = fp.readlines()
    above_first_section = True
    # Preprocess config file to preserve comments
    for i, line in enumerate(lines):
        if line.startswith("["):
            above_first_section = False
        elif line.startswith(self._comment_prefixes_backup):
            if above_first_section:
                # Remove this line for now
                lines[i] = ""
                self._top_comments.append(line)
            else:
                # Store comment as value with unique key based on line number
                lines[i] = self._comment_template.format(i, line)

    # Feed the preprocessed file to the original _read method
    return super()._read(io.StringIO("".join(lines)), fpname)

def write(self, fp, space_around_delimiters=True):
    # Write the config to an in-memory file
    with io.StringIO() as sfile:
        super().write(sfile, space_around_delimiters)
        # Start from the beginning of sfile
        sfile.seek(0)
        lines = sfile.readlines()

    for i, line in enumerate(lines):
        # Remove the comment id prefix
        lines[i] = self._comment_regex.sub("", line, 1)

    fp.write("".join(self._top_comments + lines))

Stoller