What happens when I clone a repository with symlinks on Windows?

Question

There's been a lot of questions about adding support for symlinks on Windows. But, what actually happens when I clone a repository with symlinks on Windows?

Carin

Modern NTFS actually has real symlinks .. and hardlinks .. and directory junctions. No idea, though :) – Scully

I would assume they either work, or they just look like some unusable file? – Urba

Creating symlinks on Windows still doesn't work natively, even as of Git for Windows v2.3.6.. – Cutcheon

Answer 1

Since version 1.5.3 of the native Git client git clone and git init will probe the target file system for symlink support and set the local repository configuration for core.symlinks accordingly, i.e. to false for FAT or NTFS. This makes symlinks created and committed e.g. under Linux appear as plain text files that contain the link text under Windows (see the git config documentation on core.symlinks for details).

Since Git for Windows version 2.10.2 the installer has an explicit option to enable symbolic link support.

In older versions of Git for Windows you can manually set core.symlinks to true which enabled Git to create symbolic links under the following constraints:

• Symbolic links are only available on Windows Vista and later.
• Symbolic links will only work on NTFS, not on FAT.
• You need to be an admin and / or have the SeCreateSymbolicLinkPrivilege privilege.
• Symbolic links on remote filesystems are disabled by default.
• Windows' symbolic links are typed.
• Many programs do not understand symbolic links (that includes older version of Windows Explorer).

More details are available in the Git for Windows wiki.

In older versions of Git for Windows manually setting core.symlinks manually to true after cloning and reset your working tree, you would get error messages similar to

$ git reset --hard HEAD
error: unable to create symlink directory (Function not implemented)
error: unable to create symlink linux-links/this_is_a_symbolic_link_to_file (Function not implemented)
fatal: Could not reset index file to revision 'HEAD'.

As a side note, the JGit client did not probe the target file system for symlink support until its version 3.3, so the core.symlinks setting was falling back to whatever the system / global Git configuration was. Starting with version 3.3 JGit probes for symlink support but seems to be too conservative, setting core.symlinks = false in some cases where symlinks would be in fact supported.

You can checkout https://github.com/sschuberth/git-playground which contains a bunch of links created on Linux for testing.

Apograph

See also this pull request about adding some support for symlinks to Git. – Zee

I found the now removed links to NTFS symbolic link restrictions @ WP and Why is SeCreateSymbolicLinkPrivilege ignored on Windows 8? @ SO useful. – Gossipry

I'm using win10 and git 2.11.0. When I run git clone --config core.symlinks=true https://github.com/sschuberth/sandbox.git the directory symlink and this_is_a_symbolic_link_to_file are both completely missing. When using core.symlinks=false the files are there but obivously not working like proper symlinks. I am able to create symlinks manually using mklink so it seems I have the proper rights. Does anyone have an idea on why the links are not checked out / genereated? – Safekeeping

Are you really using git.exe from Git for Windows, or maybe accidentally another one, like from Cygwin? – Apograph

So if core.symlinks=true, cloning a git repository that has git symlinks will result in windows symlinks, and committing and pushing windows symlinks will convert them to git symlinks? – Platysma

Yes, given that you use the latest Git for Windows and the constraints mentioned in my answer are met. – Apograph

I don't have core.symlinks=true on my git. I'm using Cygwin git, which means its compiled under cygwin environment. When I create cygwin symlinks (not ntfs windows symlinks), they are committed and pushed as git symlinks, and when I clone the directory back, they are created as cygwin symlinks. Although cygwin does have an option to allow creation of ntfs symlinks as cygwin symlinks, this does mean that it may be difficult to support the usage of both cygwin symlinks and ntfs symlinks in the same Git repository. – Platysma

True, but only provided that people use Cygwin Git under Windows, which to my experience most Windows users don't, as the official "Git for Windows" is based on MSYS2 and works differently in this regard. – Apograph

@ArnovanOordt I turned on developer mode in Windows 10 (to allow non-admin symlinks), set core.symlinks to true, and cloned your repo, and everything works as expected. The directory and file symlinks are created in the repo. Thanks for the demo repo! – Whiny

Answer 2

One solution would have a filter in order to detect symlinks stored by Git and replace them with Windows symlink.
That is detailed in "Git Symlinks in Windows"

However, true symlink support isn't just for now:
See issue 224 and the very recent (July 2012) discussion on GitHub (which you looked at):

here are three types of file system links on Windows: hardlinks, junctions, and symlinks.

• Hardlinks and junctions are available since NT. Hardlinks can only point to files, junctions only to directories (on the same volume).
• The symlinks available since Vista can point to either files or directories, also on different volumes.
• mklink, which ships since Vista, can create all of the above. But the way it is called in the script makes it only create symlinks (which is good, IMHO, as they most closely resemble Linux symbolic links).

For pre-Vista, we'd need a fallback that either creates hardlinks for files using "fsutil hardlink" (but probably only if "ln" is called without "-s") and creates junctions for directories using "fsutils reparsepoint", or simply calls the original ln.exe.

In addition to breaking Windows XP setups, a change like this will also break standard Windows 7 setups, because mklink requires administrator privileges by default. This can be fixed by checking if it worked or not, and reverting to copying in such cases.

Just for the record: I played around a bit with trying to make symlink support in Git for Windows itself recently, but ended up concluding that the "symlink support" in Windows 7 and up is pretty close to useless for emulating Unix symlinks.

There is a project claiming "Open Source, 100% Compatible ln for Windows (and Junction Point library)", but:

Unfortunately, normal users doesn't have the required permissions to create symlinks by default on Windows. Combine this with the fact that you cannot change what a symlink points to in the same way that POSIX requires, makes them more or less useless for us. This I already wrote above.

Now, the authors can claim all they want that this is "100% compatible" for all I care, but a quick look at the source code reveals that it's not. They do not provide any fallbacks, they don't even load the CreateSymbolicLink function dynamically. So the result on non-symlink capable Windows versions will be a crash with a missing symbol error.

Hendrika

Note that the GitHub issue you're linking to is not about symbolic links that are checked into a Git repository (which is what the OP asks about), but about making ln included in msysGit (the development environment to build Git for Windows) to create Windows Vista symlinks. – Apograph

@Apograph true, but it is at the origin of a more general discussion about symlink which was interesting for the problem at hand. The SO question I link to offer an alternative to text symlinks. The true issue, though, remains the need for administrative privileges for using symlink... that is a deal-breaker, certainly in corp environment. – Hendrika

schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html is a Windows Explorer shell extions for Symlinks, hard likes etc. Is good! – Lawley

Answer 3

I have been working on Symlink support in msysgit here:

https://github.com/frogonwheels/git (branch mrg/symlink-v* .. currently v2)

The tests do not run to completion yet, and I have limited time in which to work on it, and no real short-term goal to motivate me. It'd be nice to be able to use projects like git-annex under msysgit.

My work is hampered by the lack of symlink support in the msys shell as well.

There's a command-line for granting the privileges that is suggested by the cygwin ln command. (You will need to run this as administrator).

editrights -a SeCreateSymbolicLinkPrivilege -a $YOUR_USER

The whole problem of Directory vs file symlinks is a biggie.

At the moment I'm of the opinion that as much as we can, we limit ourselves to making File symlinks work... and not allow directory symlinks in msysgit. It's not ideal, but the reality is that any solution is a bit of a cludge, trying to impose possix linking onto the realities of the NTFS incompatibilities with possix linking is just painful.

We can try and detect whether the target is file or directory, but I can think of a few issues with that just off the top of my head - especially the whole problem of which order the entities get created.

Scopas

About creating symlinks from msysgit, you can do it directly from the terminal (assuming msys was called as Administrator) if you call cmd.exe or powershell, which are conveniently in the PATH, then calling mklink in cmd.exe or some other command in powershell. This doesn't make Linux symlinks, and they likely won't work for Git, I just felt it had to be said. – Burbank

Also, why are NTFS file vs folder symlinks a "problem"? – Burbank

Thanks for all your work on this! I must be missing something though because I tried compiling both your fork (branch mrg/symlink-v6) and the 2.0.1 version of msys/msysgit, and in both cases I still get "unable to create symlink <file> (Function not implemented)" when cloning a repo with symlinks. – Evaluate

@trysis: on NTFS, if you create a file symlink, then it can only point to a file. So if you remove the target file and replace it with a directory it the symlink is broken. On POSIX, the symlink will still work, because POSIX symlinks do not distinguish between file vs directory. – Letterperfect