Symlinks on windows?

Question

Does anyone know of a way to make/read symbolic links across versions of win32 from Python? Ideally there should be a minimum amount of platform specific code, as I need my app to be cross platform.

Milka

The functionality I need is to be able to create a directory containing links to files from disparate places in the file system, and then have my Python code be able to open those files as though they were in that directory. – Milka

Why do you need that functionality? – Trypsin

Why not just use os.symlink? Works for me on Windows 10 (needs to be run as Administrator). – Drabble

Answer 1

NTFS file system has junction points, I think you may use them instead, You can use python win32 API module for that e.g.

import win32file

win32file.CreateSymbolicLink(fileSrc, fileTarget, 1)

If you do not want to rely on win32API module, you can always use ctypes and directly call CreateSymbolicLink win32 API e.g.

import ctypes

kdll = ctypes.windll.LoadLibrary("kernel32.dll")

kdll.CreateSymbolicLinkA("d:\\test.txt", "d:\\test_link.txt", 0)

MSDN (http://msdn.microsoft.com/en-us/library/aa363866(VS.85).aspx) says Minimum supported client is Windows Vista

In addition: This also works with directories (indicate that with the third argument). With unicode support it looks like this:

kdll.CreateSymbolicLinkW(UR"D:\testdirLink", UR"D:\testdir", 1)

also see Create NTFS junction point in Python

Tetherball

You shouldn't use the ..A Windows API, use ..W (Unicode) instead - every time - even in examples such as this. – Pastrami

The parameter names shown for win32file.CreateSymbolicLink are a little confusing. For those who are wondering, the first is the name of the link to create, the second is the path that it should link to. – Vertumnus

Just for clarity: the last parameter on win32file.CreateSymboleLink should be 1 for directories, 0 for files. – Vertumnus

Every single Windows version that supports this function will load kernel32.dll into the process anyway. No need to make this explicit. – Abdul

Just to clarify: this creates a symbolic link, not a junction point – Torrent

I had some issues with using ctypes for this, see this. – Demented

Unfortunately the win32file module doesn't have any official documentation (that I could find). However, the ActiveState community has some (possibly outdated) documentation for win23file.CreateSymbolicLink: docs.activestate.com/activepython/3.2/pywin32/… – Radiotelegram

Answer 2

os.symlink works on Python 3.3 using Windows 8.1 with an NTFS filesystem.

Pharsalus

not for directory links, but yeah, it's ok – Magnetoelectricity

But it requires admin privileges: "WinError 1314: A required privilege is not held by the client." – Sheridansherie

I think that there's no getting around this - it's an OS restriction – Pharsalus

The documentation on os.symlink lists two exceptions to needing admin privileges: Having the specific SeCreateSymbolicLinkPrivilege or being a in developer mode on "newer" version on Windows 10 – Maighdlin

Answer 3

Using mklink command in subprocess create link.

from subprocess import call
call(['mklink', 'LINK', 'TARGET'], shell=True)

Unzip

I had problems with the other solutions here - but this works fine. – Demented

Answer 4

python ntfslink extension

Or if you want to use pywin32, you can use the previously stated method, and to read, use:

from win32file import *
from winioctlcon import FSCTL_GET_REPARSE_POINT

__all__ = ['islink', 'readlink']

# Win32file doesn't seem to have this attribute.
FILE_ATTRIBUTE_REPARSE_POINT = 1024
# To make things easier.
REPARSE_FOLDER = (FILE_ATTRIBUTE_DIRECTORY | FILE_ATTRIBUTE_REPARSE_POINT)

# For the parse_reparse_buffer function
SYMBOLIC_LINK = 'symbolic'
MOUNTPOINT = 'mountpoint'
GENERIC = 'generic'

def islink(fpath):
    """ Windows islink implementation. """
    if GetFileAttributes(fpath) & REPARSE_FOLDER == REPARSE_FOLDER:
        return True
    return False


def parse_reparse_buffer(original, reparse_type=SYMBOLIC_LINK):
    """ Implementing the below in Python:

    typedef struct _REPARSE_DATA_BUFFER {
        ULONG  ReparseTag;
        USHORT ReparseDataLength;
        USHORT Reserved;
        union {
            struct {
                USHORT SubstituteNameOffset;
                USHORT SubstituteNameLength;
                USHORT PrintNameOffset;
                USHORT PrintNameLength;
                ULONG Flags;
                WCHAR PathBuffer[1];
            } SymbolicLinkReparseBuffer;
            struct {
                USHORT SubstituteNameOffset;
                USHORT SubstituteNameLength;
                USHORT PrintNameOffset;
                USHORT PrintNameLength;
                WCHAR PathBuffer[1];
            } MountPointReparseBuffer;
            struct {
                UCHAR  DataBuffer[1];
            } GenericReparseBuffer;
        } DUMMYUNIONNAME;
    } REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER;

    """
    # Size of our data types
    SZULONG = 4 # sizeof(ULONG)
    SZUSHORT = 2 # sizeof(USHORT)

    # Our structure.
    # Probably a better way to iterate a dictionary in a particular order,
    # but I was in a hurry, unfortunately, so I used pkeys.
    buffer = {
        'tag' : SZULONG,
        'data_length' : SZUSHORT,
        'reserved' : SZUSHORT,
        SYMBOLIC_LINK : {
            'substitute_name_offset' : SZUSHORT,
            'substitute_name_length' : SZUSHORT,
            'print_name_offset' : SZUSHORT,
            'print_name_length' : SZUSHORT,
            'flags' : SZULONG,
            'buffer' : u'',
            'pkeys' : [
                'substitute_name_offset',
                'substitute_name_length',
                'print_name_offset',
                'print_name_length',
                'flags',
            ]
        },
        MOUNTPOINT : {
            'substitute_name_offset' : SZUSHORT,
            'substitute_name_length' : SZUSHORT,
            'print_name_offset' : SZUSHORT,
            'print_name_length' : SZUSHORT,
            'buffer' : u'',
            'pkeys' : [
                'substitute_name_offset',
                'substitute_name_length',
                'print_name_offset',
                'print_name_length',
            ]
        },
        GENERIC : {
            'pkeys' : [],
            'buffer': ''
        }
    }

    # Header stuff
    buffer['tag'] = original[:SZULONG]
    buffer['data_length'] = original[SZULONG:SZUSHORT]
    buffer['reserved'] = original[SZULONG+SZUSHORT:SZUSHORT]
    original = original[8:]

    # Parsing
    k = reparse_type
    for c in buffer[k]['pkeys']:
        if type(buffer[k][c]) == int:
            sz = buffer[k][c]
            bytes = original[:sz]
            buffer[k][c] = 0
            for b in bytes:
                n = ord(b)
                if n:
                    buffer[k][c] += n
            original = original[sz:]

    # Using the offset and length's grabbed, we'll set the buffer.
    buffer[k]['buffer'] = original
    return buffer

def readlink(fpath):
    """ Windows readlink implementation. """
    # This wouldn't return true if the file didn't exist, as far as I know.
    if not islink(fpath):
        return None

    # Open the file correctly depending on the string type.
    handle = CreateFileW(fpath, GENERIC_READ, 0, None, OPEN_EXISTING, FILE_FLAG_OPEN_REPARSE_POINT, 0) \
                if type(fpath) == unicode else \
            CreateFile(fpath, GENERIC_READ, 0, None, OPEN_EXISTING, FILE_FLAG_OPEN_REPARSE_POINT, 0)

    # MAXIMUM_REPARSE_DATA_BUFFER_SIZE = 16384 = (16*1024)
    buffer = DeviceIoControl(handle, FSCTL_GET_REPARSE_POINT, None, 16*1024)
    # Above will return an ugly string (byte array), so we'll need to parse it.

    # But first, we'll close the handle to our file so we're not locking it anymore.
    CloseHandle(handle)

    # Minimum possible length (assuming that the length of the target is bigger than 0)
    if len(buffer) < 9:
        return None
    # Parse and return our result.
    result = parse_reparse_buffer(buffer)
    offset = result[SYMBOLIC_LINK]['substitute_name_offset']
    ending = offset + result[SYMBOLIC_LINK]['substitute_name_length']
    rpath = result[SYMBOLIC_LINK]['buffer'][offset:ending].replace('\x00','')
    if len(rpath) > 4 and rpath[0:4] == '\\??\\':
        rpath = rpath[4:]
    return rpath

def realpath(fpath):
    from os import path
    while islink(fpath):
        rpath = readlink(fpath)
        if not path.isabs(rpath):
            rpath = path.abspath(path.join(path.dirname(fpath), rpath))
        fpath = rpath
    return fpath


def example():
    from os import system, unlink
    system('cmd.exe /c echo Hello World > test.txt')
    system('mklink test-link.txt test.txt')
    print 'IsLink: %s' % islink('test-link.txt')
    print 'ReadLink: %s' % readlink('test-link.txt')
    print 'RealPath: %s' % realpath('test-link.txt')
    unlink('test-link.txt')
    unlink('test.txt')

if __name__=='__main__':
    example()

Adjust the attributes in the CreateFile to your needs, but for a normal situation, it should work. Feel free to improve on it.

It should also work for folder junctions if you use MOUNTPOINT instead of SYMBOLIC_LINK.

You may way to check that

sys.getwindowsversion()[0] >= 6

if you put this into something you're releasing, since this form of symbolic link is only supported on Vista+.

Pursy

I submitted an edit to this but it was rejected even though there is in fact a bug present. In islink(), the masked value must be compared to the mask to avoid false-positives (the zero = false/nonzero = true shortcut would only with a single-bit mask, e.g., 1024). Otherwise normal dirs are also identified as "links". The line should read "if GetFileAttributes(fpath) & REPARSE_FOLDER == REPARSE_FOLDER:" – Marcoux

If I did reject it, I must've done so by mistake and I'm sorry about that. (To be honest, I'm not quite sure where I would even go to see edit requests, given I've never received one on here). I went ahead and updated with your fix, though, so thank you for that. – Pursy

Oh no, you didn't reject anything... two other editors did, which was peculiar since my suggested edit simply corrected the bug. That said, thanks again for your post - it was very helpful and worth delving into! :-) – Marcoux

Keep in mind that this implementation of islink will return True if the path does not exist. – Iyeyasu

Answer 5

Problem is, as explained e.g. here, that Windows' own support for the functionality of symbolic links varies across Windows releases, so that e.g. in Vista (with lots of work) you can get more functionality than in XP or 2000 (nothing AFAIK on other win32 versions). Or you could have shortcuts instead, which of course have their own set of limitations and aren't "really" equivalent to Unix symbolic links. So, you have to specify exactly what functionalities you require, how much of those you are willing to sacrifice on the altar of cross-win32 operation, etc -- THEN, we can work out how to implement the compromise you've chosen in terms of ctypes or win32all calls... that's the least of it, in a sense.

Mithridate

Answer 6

I put the following into Lib/site-packages/sitecustomize.py

import os

__CSL = None
def symlink(source, link_name):
    '''symlink(source, link_name)
       Creates a symbolic link pointing to source named link_name'''
    global __CSL
    if __CSL is None:
        import ctypes
        csl = ctypes.windll.kernel32.CreateSymbolicLinkW
        csl.argtypes = (ctypes.c_wchar_p, ctypes.c_wchar_p, ctypes.c_uint32)
        csl.restype = ctypes.c_ubyte
        __CSL = csl
    flags = 0
    if source is not None and os.path.isdir(source):
        flags = 1
    if __CSL(link_name, source, flags) == 0:
        raise ctypes.WinError()

os.symlink = symlink

Nyctalopia

Answer 7

Juntalis's code does not handle Unicode so I modified it to use ctypes and also simplified it using struct. I've also consulted the code from Using a struct as a function argument with the python ctypes module

import os, ctypes, struct
from ctypes import windll, wintypes

FSCTL_GET_REPARSE_POINT = 0x900a8

FILE_ATTRIBUTE_READONLY      = 0x0001
FILE_ATTRIBUTE_HIDDEN        = 0x0002
FILE_ATTRIBUTE_DIRECTORY     = 0x0010
FILE_ATTRIBUTE_NORMAL        = 0x0080
FILE_ATTRIBUTE_REPARSE_POINT = 0x0400


GENERIC_READ  = 0x80000000
GENERIC_WRITE = 0x40000000
OPEN_EXISTING = 3
FILE_READ_ATTRIBUTES = 0x80
FILE_FLAG_OPEN_REPARSE_POINT = 0x00200000
INVALID_HANDLE_VALUE = wintypes.HANDLE(-1).value

INVALID_FILE_ATTRIBUTES = 0xFFFFFFFF

FILE_FLAG_OPEN_REPARSE_POINT = 2097152
FILE_FLAG_BACKUP_SEMANTICS = 33554432
# FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTI
FILE_FLAG_REPARSE_BACKUP = 35651584


GetFileAttributes = windll.kernel32.GetFileAttributesW
_CreateFileW = windll.kernel32.CreateFileW
_DevIoCtl = windll.kernel32.DeviceIoControl
_DevIoCtl.argtypes = [
    wintypes.HANDLE, #HANDLE hDevice
    wintypes.DWORD, #DWORD dwIoControlCode
    wintypes.LPVOID, #LPVOID lpInBuffer
    wintypes.DWORD, #DWORD nInBufferSize
    wintypes.LPVOID, #LPVOID lpOutBuffer
    wintypes.DWORD, #DWORD nOutBufferSize
    ctypes.POINTER(wintypes.DWORD), #LPDWORD lpBytesReturned
    wintypes.LPVOID] #LPOVERLAPPED lpOverlapped
_DevIoCtl.restype = wintypes.BOOL


def islink(path):
    assert os.path.isdir(path), path
    if GetFileAttributes(path) & FILE_ATTRIBUTE_REPARSE_POINT:
        return True
    else:
        return False


def DeviceIoControl(hDevice, ioControlCode, input, output):
    # DeviceIoControl Function
    # http://msdn.microsoft.com/en-us/library/aa363216(v=vs.85).aspx
    if input:
        input_size = len(input)
    else:
        input_size = 0
    if isinstance(output, int):
        output = ctypes.create_string_buffer(output)
    output_size = len(output)
    assert isinstance(output, ctypes.Array)
    bytesReturned = wintypes.DWORD()
    status = _DevIoCtl(hDevice, ioControlCode, input,
                       input_size, output, output_size, bytesReturned, None)
    print "status(%d)" % status
    if status != 0:
        return output[:bytesReturned.value]
    else:
        return None


def CreateFile(path, access, sharemode, creation, flags):
    return _CreateFileW(path, access, sharemode, None, creation, flags, None)


SymbolicLinkReparseFormat = "LHHHHHHL"
SymbolicLinkReparseSize = struct.calcsize(SymbolicLinkReparseFormat);

def readlink(path):
    """ Windows readlink implementation. """
    # This wouldn't return true if the file didn't exist, as far as I know.
    assert islink(path)
    assert type(path) == unicode

    # Open the file correctly depending on the string type.
    hfile = CreateFile(path, GENERIC_READ, 0, OPEN_EXISTING,
                       FILE_FLAG_REPARSE_BACKUP)
    # MAXIMUM_REPARSE_DATA_BUFFER_SIZE = 16384 = (16*1024)
    buffer = DeviceIoControl(hfile, FSCTL_GET_REPARSE_POINT, None, 16384)
    CloseHandle(hfile)

    # Minimum possible length (assuming length of the target is bigger than 0)
    if not buffer or len(buffer) < 9:
        return None

    # Parse and return our result.
    # typedef struct _REPARSE_DATA_BUFFER {
    #   ULONG  ReparseTag;
    #   USHORT ReparseDataLength;
    #   USHORT Reserved;
    #   union {
    #       struct {
    #           USHORT SubstituteNameOffset;
    #           USHORT SubstituteNameLength;
    #           USHORT PrintNameOffset;
    #           USHORT PrintNameLength;
    #           ULONG Flags;
    #           WCHAR PathBuffer[1];
    #       } SymbolicLinkReparseBuffer;
    #       struct {
    #           USHORT SubstituteNameOffset;
    #           USHORT SubstituteNameLength;
    #           USHORT PrintNameOffset;
    #           USHORT PrintNameLength;
    #           WCHAR PathBuffer[1];
    #       } MountPointReparseBuffer;
    #       struct {
    #           UCHAR  DataBuffer[1];
    #       } GenericReparseBuffer;
    #   } DUMMYUNIONNAME;
    # } REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER;

    # Only handle SymbolicLinkReparseBuffer
    (tag, dataLength, reserver, SubstituteNameOffset, SubstituteNameLength,
     PrintNameOffset, PrintNameLength,
     Flags) = struct.unpack(SymbolicLinkReparseFormat,
                            buffer[:SymbolicLinkReparseSize])
    print tag, dataLength, reserver, SubstituteNameOffset, SubstituteNameLength
    start = SubstituteNameOffset + SymbolicLinkReparseSize
    actualPath = buffer[start : start + SubstituteNameLength].decode("utf-16")
    # This utf-16 string is null terminated
    index = actualPath.find(u"\0")
    assert index > 0
    if index > 0:
        actualPath = actualPath[:index]
    if actualPath.startswith(u"?\\"):
        return actualPath[2:]
    else:
        return actualPath

Startle

Yeah, didn't realize at the time of writing it that the reparse points were stored as unicode strings. (Stupid mistake to be sure) You're much better off using the above code, or a module I just recently found on github. – Pursy

Answer 8

As mentioned in another answer, using subprocess.call is likely the best option for windows. However calling 'mklink' directly may result in:

[WinError 2] The system cannot find the file specified

On Windows Server 2016, I was able to get the following to work for files:

import subprocess
subprocess.call(['cmd', '/c', 'mklink', '<path_for_symlink>', '<path_for_file>'])

Change the switches above as per mklink docs.

Pontiac

Answer 9

Trying to create a Symlink in Windows I always got the error

A required privilege is not held by the client

However I was successfull when creating a shortcut with this code

import win32com.client
import pythoncom
import os

def create_shortcut(original_filepath, shortcut_filepath):
    shell = win32com.client.Dispatch("WScript.Shell")
    shortcut = shell.CreateShortCut(shortcut_filepath)
    shortcut.Targetpath = original_filepath
    shortcut.WindowStyle = 7
    shortcut.save()

create_shortcut(r'C:\Users\xxx\Desktop\test.jpg', 
                r'C:\Users\xxx\Desktop\test.lnk')

Note: make sure the shortcut ends with '.lnk'

Sedan

Answer 10

here is the link containing all methods of kernel32.dll

http://www.geoffchappell.com/studies/windows/win32/kernel32/api/

I used CreateHardLinkA on Windows xp sp3, it worked!

import ctypes if os.path.exists(link_file): os.remove(link_file)

dll = ctypes.windll.LoadLibrary("kernel32.dll")
dll.CreateHardLinkA(link_file, _log_filename, 0)

Mainly

Symlinks as they were introduced in Vista are implemented as reparse points and thus closer to symlinks in POSIX. Hardlinks got nothing to do with that. – Abdul

Answer 11

python3 supports symlinks on windows vista+(windows 10...) just need to run in an elevated cmd

Maecenas