Prepend a line to an existing file in Python

Question

I need to add a single line to the first line of a text file and it looks like the only options available to me are more lines of code than I would expect from python. Something like this:

f = open('filename','r')
temp = f.read()
f.close()

f = open('filename', 'w')
f.write("#testfirstline")

f.write(temp)
f.close()

Is there no easier way? Additionally, I see this two-handle example more often than opening a single handle for reading and writing ('r+') - why is that?

Brottman

(Worth noting: you're probably better off reading the file line by line and writing to a temp. file. When you're done, remove the original file and replace with the temp.) – Gary

Answer 1

Python makes a lot of things easy and contains libraries and wrappers for a lot of common operations, but the goal is not to hide fundamental truths.

The fundamental truth you are encountering here is that you generally can't prepend data to an existing flat structure without rewriting the entire structure. This is true regardless of language.

There are ways to save a filehandle or make your code less readable, many of which are provided in other answers, but none change the fundamental operation: You must read in the existing file, then write out the data you want to prepend, followed by the existing data you read in.

By all means save yourself the filehandle, but don't go looking to pack this operation into as few lines of code as possible. In fact, never go looking for the fewest lines of code -- that's obfuscation, not programming.

Harberd

"never go looking for the fewest lines of code -- that's obfuscation, not programming" - Hiding the amount of time that it takes to perform a function is not obfuscation, it is abstraction. If the purpose of code was to take a proportional amount of time to read as to run, code would have an entirely different structure than it actually does. – Chapple

"You must read in the existing file, then write out the data you want to prepend, followed by the existing data you read in" - this is not really true, it can be much more memory-efficient to chunk & interleave the reading & writing operations so that the full contents of the file never need to reside in memory at once. Dealing with the details of this would in fact make an excellent library. – Stupa

Answer 2

I would stick with separate reads and writes, but we certainly can express each more concisely:

with open('filename', 'r') as original:
    data = original.read()
with open('filename', 'w') as modified:
    modified.write("new first line\n" + data)

Selfcontrol

Answer 3

Other approach:

with open("infile") as f1:
    with open("outfile", "w") as f2:
        f2.write("#test firstline")
        for line in f1:
            f2.write(line)

or a one liner:

open("outfile", "w").write("#test firstline\n" + open("infile").read())

Thanks for the opportunity to think about this problem :)

Cheers

Warford

Answer 4

with open("file", "r+") as f: s = f.read(); f.seek(0); f.write("prepend\n" + s)

Yumuk

Answer 5

You can save one write call with this:

f.write('#testfirstline\n' + temp)

When using 'r+', you would have to rewind the file after reading and before writing.

Marcelmarcela

+1, although you could have saved a whole line of code by doing f.write('#testfirstline\n' + temp) – Orphrey

I would recommend f.writelines(('#testfirstline\n',tmp)). Then if temp is huge you aren't creating another huge string in order to write this all out. Or, just use the extra write line as in the OP... – Abagail

Answer 6

Here's a 3 liner that I think is clear and flexible. It uses the list.insert function, so if you truly want to prepend to the file use l.insert(0, 'insert_str'). When I actually did this for a Python Module I am developing, I used l.insert(1, 'insert_str') because I wanted to skip the '# -- coding: utf-8 --' string at line 0. Here is the code.

f = open(file_path, 'r'); s = f.read(); f.close()
l = s.splitlines(); l.insert(0, 'insert_str'); s = '\n'.join(l)
f = open(file_path, 'w'); f.write(s); f.close()

Adjure

You could make it into a 1-liner too – Action

this one prints out chinese characters! – Yaya

Hi Ulf Gjerdingen! Does the file contain chinese characters? – Adjure

Answer 7

This does the job without reading the whole file into memory, though it may not work on Windows

def prepend_line(path, line):
    with open(path, 'r') as old:
        os.unlink(path)
        with open(path, 'w') as new:
            new.write(str(line) + "\n")
            shutil.copyfileobj(old, new)

Yalta

I like this, though you have to be careful as it will result in data loss if interrupted. – Romaromagna

Doing this will overwrite the file. Is it possible to create a new file using shutil.copyfileobj instead of overwriting? – Spin

Sorry @Spin I don't quite understand what you mean. If you want to write to a new file just put a different path in the second call to open. – Yalta

Answer 8

One possibility is the following:

import os
open('tempfile', 'w').write('#testfirstline\n' + open('filename', 'r').read())
os.rename('tempfile', 'filename')

Testaceous

This is unsafe on POSIX (race conditions on some filesystems, XFS I believe). One needs to call f.flush() and os.fsync(f.fileno()) on the temporary file before the rename. – Jamima

thunk.org/tytso/blog/2009/03/12/… – Jamima

Answer 9

If you wish to prepend in the file after a specific text then you can use the function below.

def prepend_text(file, text, after=None):
    ''' Prepend file with given raw text '''
    f_read = open(file, 'r')
    buff = f_read.read()
    f_read.close()
    f_write = open(file, 'w')
    inject_pos = 0
    if after:
        pattern = after
        inject_pos = buff.find(pattern)+len(pattern)
    f_write.write(buff[:inject_pos] + text + buff[inject_pos:])
    f_write.close()

So first you open the file, read it and save it all into one string. Then we try to find the character number in the string where the injection will happen. Then with a single write and some smart indexing of the string we can rewrite the whole file including the injected text now.

Nymphalid

Answer 10

Am I not seeing something or couldn't we just use a buffer large-enough to read-in the input file in parts (instead of the whole content) and with this buffer traverse the file while it is open and keep exchanging file<->buffer contents?

This seems much more efficient (for big files especially) than reading the whole content in memory, modifying it in memory and writing it back to the same file or (even worse) a different one. Sorry that now I don't have time to implement a sample snippet, I'll get back to this later, but maybe you get the idea.

Elwira

Answer 11

As I suggested in this answer, you can do it using the following:

def prepend_text(filename: Union[str, Path], text: str):
    with fileinput.input(filename, inplace=True) as file:
        for line in file:
            if file.isfirstline():
                print(text)
            print(line, end="")

Illbred

Answer 12

If you rewrite it like this:

with open('filename') as f:
    read_data = f.read()
with open('filename', 'w') as f:
    f.write("#testfirstline\n" + read_data)

It's rather short and simple. For 'r+' the file needs to exist already.

Poul

Answer 13

this worked for me

def prepend(str, file):
    with open(file, "r") as fr:
        read = fr.read()
        with open(file, "w") as fw:
            fw.write(str + read)
            fw.close()

Bizarre

Answer 14

What about using OS library.

In this particular case with OS Linux and using the sed function

import os

header = 'Your first line'
cmd_header = (r"sed -i '1s/^/" + f"{header}" + r"\n/'" + f" {file_name}")
os.cmd(cmd_header)

This will create a new line into the specified file_name location and insert the header string. This process does not rewrite the whole file structure and can process big files.

Hurty

Answer 15

Use pathlib.Path read_text() and write_text() to prepend to a file concisely:

from pathlib import Path

file = Path("filename")
original = file.read_text()
file.write_text(f"new first line\n{original}")

Spectacle