Modifying a symlink in python

Question

How do I change a symlink to point from one file to another in Python?

The os.symlink function only seems to work to create new symlinks.

Betseybetsy

Answer 1

If you need an atomic modification, unlinking won't work.

A better solution would be to create a new temporary symlink, and then rename it over the existing one:

os.symlink(target, tmpLink)
os.rename(tmpLink, linkName)

You can check to make sure it was updated correctly too:

if os.path.realpath(linkName) == target:
    # Symlink was updated

According to the documentation for os.rename though, there may be no way to atomically change a symlink in Windows. In that case, you would just delete and re-create.

Claymore

Python 3.3 has os.replace which can be used instead of os.rename to provide the same behaviour on Posix & Windows systems. – Bleier

Won't work on Linux systems - needs replace. See this answer for a complete solution (including generating a temporary symlink pathname). – Ybarra

@TomHale, this works fine on my Linux system provided the original file you're overriding is also a symlink (true for me). And unlike your solution it works on Python 2.7. – Gomez

Note that 1) replace raises IsADirectoryError if linkName is a directory and tmpLink will then need to be deleted. 2) replace may fail if the two files are not on the same filesystem. I deal with both cases in this answer. – Ybarra

Answer 2

A little function for Python3 which tries to symlink and if it fails because of an existing file, it removes it and links again.

import os, errno

def symlink_force(target, link_name):
    try:
        os.symlink(target, link_name)
    except OSError as e:
        if e.errno == errno.EEXIST:
            os.remove(link_name)
            os.symlink(target, link_name)
        else:
            raise e

Fishtail

Has a race condition: the known link_name could be created again between removal and symlink creation. Try atomically overwriting the existing symlink. – Ybarra

@TomHale You are right that my solution can fail to do the job. Funny is that your solution can fail, too. ;-) – Fishtail

While my answer did point out its race condition, you're right that my comment didn't highlight it. I've now fixed another race in my answer. There is still the very unlikely case that a file is created by another process at a randomly generated pathname. Thanks for your comment on my answer, too. – Ybarra

Would you consider updating to use except FileExistsError:? I got a syntax error as written. – Ybarra

Thanks; with "except OSError as e" it worked for me on RPi Python 3.7.3. – Reaper

Answer 3

Given overwrite=True, this function will safely overwrite an existing file with a symlink.

It is cognisant of race conditions, which is why it is not short, but it is safe.

import os, tempfile

def symlink(target, link_name, overwrite=False):
    '''
    Create a symbolic link named link_name pointing to target.
    If link_name exists then FileExistsError is raised, unless overwrite=True.
    When trying to overwrite a directory, IsADirectoryError is raised.
    '''

    if not overwrite:
        os.symlink(target, link_name)
        return

    # os.replace() may fail if files are on different filesystems
    link_dir = os.path.dirname(link_name)

    # Create link to target with temporary filename
    while True:
        temp_link_name = tempfile.mktemp(dir=link_dir)

        # os.* functions mimic as closely as possible system functions
        # The POSIX symlink() returns EEXIST if link_name already exists
        # https://pubs.opengroup.org/onlinepubs/9699919799/functions/symlink.html
        try:
            os.symlink(target, temp_link_name)
            break
        except FileExistsError:
            pass

    # Replace link_name with temp_link_name
    try:
        # Pre-empt os.replace on a directory with a nicer message
        if not os.path.islink(link_name) and os.path.isdir(link_name):
            raise IsADirectoryError(f"Cannot symlink over existing directory: '{link_name}'")
        os.replace(temp_link_name, link_name)
    except:
        if os.path.islink(temp_link_name):
            os.remove(temp_link_name)
        raise

Notes for pedants:

1. If the function fails (e.g. computer crashes), an additional random link to the target might exist.

2. An unlikely race condition still remains: the symlink created at the randomly-named temp_link_name could be modified by another process before replacing link_name.

I raised a python issue to highlight the issues of os.symlink() requiring the target to not exist, where I was advised to raise my suggestion on the python-ideas mailing list

Credit to Robert Siemer’s input.

Ybarra

If something “pathological” happens it is better to cycle without sleep, I would argue. Either that or raise an exception (after X iterations)... Anyway... super theoretical... this is why my solution tried only twice. – Fishtail

@RobertSiemer Thanks for the excellent edits overall. I restored my text about the possible race condition as I believe it is still valid. Please comment if you disagree. – Ybarra

You love using those words “race condition”, don’t you? ;-) An “attacking program” with the same permissions could just delete the link after it got properly created (function returned), but before the function caller can do something else. There is nothing to prevent that. I don’t see anything special in case “2.”. It is just another case of “1.”... I used different wording, didn’t I? The function might not just be forcefully “interrupted”; it can also fail for other reasons which would not be “interruptions”. – Fishtail

...so if no one interferes, this function will work. – So will my solution. The only advantage of replace() is this guarantee: link_name will either be what is was before, or it will be a link to target with no point in between. – Fishtail

Won't work for us poor people still on Python 2.7 where "os.replace()" doesn't exist – Gomez

What is the while loop for? Under what conditions would it need to repeat? – Renettarenew

@Renettarenew The loop repeats in the unlikely event that a file is created having the same name as the generated temporary filename, and before the symlink(). – Ybarra

After all this bragging about race conditions, ironically, your code has a race condition in the function tempfile.mktemp() which has been deprecated since Python 2.3 and should be no longer used. – Cursor

@Cursor there is no race from mktemp: any other process creating the file between mktemp and symlink is handled by FileExistsError. The link you post includes the word 'may'. I avoid mkstemp as it both creates and opens the file, requiring both closing then deleting the file, which must not exist (prerequisite for a successful symlink.) – Ybarra

Answer 4

You could os.unlink() it first, and then re-create using os.symlink() to point to the new target.

Astrogate

Has a race condition: the known link_name could be created again between removal and symlink creation. Try atomically overwriting the existing symlink. – Ybarra

Answer 5

I researched this question recently, and found out that the best way is indeed to unlink and then symlink. But if you need just to fix broken links, for example with auto-replace, then you can do os.readlink:

for f in os.listdir(dir):
    path = os.path.join(dir, f)
    old_link = os.readlink(path)
    new_link = old_link.replace(before, after)
    os.unlink(path)
    os.symlink(new_link, path)

Vendee

Answer 6

Don't forget to add a raise command in the case when e.errno != errno.EEXIST You don't want to hide an error then:

if e.errno == errno.EEXIST:
     os.remove(link_name)
     os.symlink(target, link_name)
else:
    raise

Paulina

Answer 7

A quick and easy solution:

while True:
     try:
         os.symlink(target, link_name)
         break
     except FileExistsError:
         os.remove(link_name)

However this has a race condition when replacing a symlink which should always exist, eg:

 /lib/critical.so -> /lib/critical.so.1.2

When upgrading by:

 my_symlink('/lib/critical.so.2.0', '/lib/critical.so')

There is a point in time when /lib/critical.so doesn't exist.

This answer avoids the race condition.

Ybarra

Answer 8

I like this version more

import os
def force_symlink(src, dst):
    if os.path.exists(dst):
        if os.path.realpath(src) == dst:
            return
        os.unlink(dst)
    os.symlink(src, dst)

Catalyst

This fails if the existing link is broken, since exists will return false if the link destination is missing. The following should also work in those cases: if os.path.islink(dst) or os.path.exists(dst): – Insincere