Does json.dump() append to file?

Question

I'm getting some unexpected behavior with json.dump(). I'm creating a file results(empty), and then using it in the code like this:

        with open(results, 'r+') as fp:
            temp = {}
            try:
                # file not empty, load existing dict, and add a key value to it
                temp = json.load(fp)
                temp[key] = value
            except json.decoder.JSONDecodeError:
                # file is empty, create a new dict 
                temp[key] = value
            # write the dictionary back into file
            json.dump(temp, fp)

If the above quote executes once, it works fine. However, if I execute it twice, I'm expecting to have a single dictionary with two keys: {key1: value1, key2: value2}, but I get instead two dictionaries: {key1: value1}{key2: value2}. What could be the reason for such behavior?

Keepsake

Answer 1

Look at the output file before and after running the code, and you should see what's going on.

Right before the json.dump, the file object points to the end of the file. You then dump data from this position.

If you try rewinding the file first, it should overwrite the data from the start:

fp.seek(0)
json.dump(temp, fp)

However, this could potentially leave dangling data beyond the first object, if it writes less data than is already in the file. I therefore suggest you restructure your code to read and write the file in two operations, wiping the file on the write. For example:

import json

filename = "foo"

print("Reading %s" % filename)
try:
    with open(filename, "rt") as fp:
        data = json.load(fp)
    print("Data: %s" % data)
except IOError:
    print("Could not read file, starting from scratch")
    data = {}

# Add some data
data["key2"] = "value2"

print("Overwriting %s" % filename)
with open(filename, "wt") as fp:
    json.dump(data, fp)

Cobbie

Answer 2

This depends on how you open the file. If you specify "w" by

with open(filename, "w") as f:
    json.dump(data,f)

the previous data will be overwritten. But if you "a" or "a+"

with open(filename, "a") as f:
    json.dump(data,f)

it would append the new data.

Ref: Python open()

Conservation

the problem with this approach is that it literally appends the data so that it ends up like {...original data...}{...appended data...} – Anemometer

it wouldn't be hard to do a f.write(",") in between or something @Anemometer Loved the movie, by the way – Possibly

Answer 3

You can do this:

with open('example.txt', 'r+') as file:

    # Read the current content
    content = file.read()

    # optionally you can print the data
    print("Before Writing:", content)
    
    # Move the file pointer to the beginning
    file.seek(0)
    
    # Write new content, which will overwrite the existing content from the beginning
    file.write("New content")
    
    # truncate the file. So if new content is shorter than the original, the extra data would be cleared.
    file.truncate()

commented lines specifies each line's purpose. Using this method, you don't need to open the file twice. Obviously, instead of file.read() & file.write(), you can use json.load() & json.dump().

Cutlor