git forces refresh index after switching between Windows and Linux

Question

I have a disk partition (format: NTFS) shared by Windows and Linux. It contains a git repository (about 6.7 GB). If I only use Windows or only use Linux to manipulate the git repository everything is okay. But everytime I switch the system, the git status command will refresh the index, which takes about 1 minute. If I run the git status in the same system again, it only take less than 1 second. Here is the result

# Just after switch from windows
[#5#wangx@manjaro:duishang_design] git status  # this command takes more than 60s
Refresh index: 100% (2751/2751), done.
On branch master
nothing to commit, working tree clean

[#10#wangx@manjaro:duishang_design] git status  # this time the command takes less than 1s
On branch master
nothing to commit, working tree clean

[#11#wangx@manjaro:duishang_design] git status  # this time the command takes less than 1s
On branch master
nothing to commit, working tree clean

I guess there is some problem with the git cache. For example: Windows and Linux all use the .git/index file as cache file, but the git in Linux system can't recognize the .git/index changed by Windows. So it can only refresh the index and replace the .git/index file, which makes the next git status super fast and git status in Windows very slow (because the Windows system will refresh the index file again).

Is my guess correct? If so, how can I set the index file for different system? How can I solve the problem?

Vacate

Experiencing this on Windows 10 Bash too – Sthenic

Answer 1

You are completely correct here:

• The thing you're using here, which Git variously calls the index, the staging area, or the cache, does in fact contain cache data.

• The cache data that it contains is the result of system calls.

• The system call data returned by a Linux system is different from the system call data returned by a Windows system.

Hence, an OS switch completely invalidates all the cache data.

... how can I use set the index file for different system?

Your best bet here is not to do this at all. Make two different work-trees, or perhaps even two different repositories. But, if that's more painful than this other alternative, try out these ideas:

The actual index file that Git uses merely defaults to .git/index. You can specify a different file by setting GIT_INDEX_FILE to some other (relative or absolute) path. So you could have .git/index-linux and .git/index-windows, and set GIT_INDEX_FILE based on whichever OS you're using.

Some Git commands use a temporary index. They do this by setting GIT_INDEX_FILE themselves. If they un-set it afterward, they may accidentally use .git/index at this point. So another option is to rename .git/index out of the way when switching OSes. Keep a .git/index-windows and .git/index-linux as before, but rename whichever one is in use to .git/index while it's in use, then rename it to .git/index-name before switching to the other system.

Again, I don't recommend attempting either of these methods, but they are likely to work, more or less.

Jennefer

Thank you, torek. I rename the .git/index file to .git/windows_index and .git/linux_index. It works fine. I tried setting GIT_INDEX_FILE environment variable, and found it will affect all my git repositories. How can I only the the git_index_file just for one certain repositry e.g. changing the .git/config file? – Vacate

@Vacate Unfortunately there's no really easy way to do that. That's one of multiple reasons I don't recommend this idea. If you're up for building Git from its source, though, you could make a patch to it that makes the default index file path add a piece of uname data. That would affect all your repositories, but would "just work" where this GIT_INDEX_FILE trick requires a lot of manual intervention. – Jennefer

Thanks. It is good enough for me to keep both the windows_index file and linux_index file. – Vacate

Do watch out for one other thing: git gc won't scan these extra index files, so any files that you git add to the (machine-specific) index needs to be committed within about 2 weeks. – Jennefer

since WSL2 has absolute trash IO on NTFS I would probably recommend different repositories – Basir

Answer 2

This might not apply to the original poster, but if Linux is being used under the Windows Subsystem for Linux (WSL), then a quick fix is use git.exe even on the Linux side. Use an alias or something to make it seamless. For example:

alias git=git.exe

Intimacy

Quick & easy solution, appears to work for me as well, git-bash on Windows side, WSL Ubuntu 18.04 on Linux side - alias git=git.exe prevents refreshing the index and appears to otherwise work identically 👍 – Honest

Note: this breaks if you're authenticating through an SSH keypair. I imagine that it's possible to set up an SSH key that will work with the Windows exe, but I'm not sure how. – Essonite

@KristenHammack your ssh keys will need to be under your windows home .ssh folder not your wsl home .ssl folder. – Intimacy

Answer 3

As torek mentioned, you probably don't want to do this. It's not generally a good idea to share a repo between operating systems.

However, it is possible, much like it's possible to share a repo between Windows and Windows Subsystem for Linux. You may want to try setting core.checkStat to minimal, and if that isn't sufficient, core.trustctime to false. That leads to the minimal amount of information being stored in the index, which means that the data is going to be as portable as possible.

Note, however, that if your repository has symlinks, that it's likely that nothing you do is going to prevent refreshes. Linux typically considers the length of a symlink to be its length in bytes, and Windows considers it to take one or more disk blocks, so there will be a mismatch in size between the operating systems. This isn't avoidable, since size is one of the attributes used in the index that can't be disabled.

Avon

Answer 4

Auto line ending setting solved my issue as in this discussion. I am referring to Windows, WSL2, Portable Linux OS, and Linux as well which I have setup and working as my work requirement. I will update in case I face any issue while preferring this approach for updating code base from different filesystems (NTFS or Linux File System).

git config --global core.autocrlf true 

Seineetmarne

Great find, but as far as I can see unrelated to the question? =} – Phrenic

It will avoid the use of different indexing if someone wish not to follow multiple indexing solution. – Seineetmarne

This is a solution to a completely different problem. Besides, enabling autocrlf will result in other problems, especially when using the same repository in Linux and Windows (as is the case in this question). – Filipino

It should not unrelated as I am referring to Windows, WSL2, Portable Linux OS, and Linux as well. With this approach, I was able to solve cross-platform problem. Been 1.5 years, I didn't face issue while working cross-platform. – Seineetmarne

Answer 5

In my case I just don't want a command that runs in a docker container (and calls git status to find out if the working directory is clean) to modify .git/index outside the container, so I'm making the command make a temporary copy of .git/index before running GIT_INDEX_FILE=my_index_copy git status.

In my case .git/index is approximately 1 MB, which is small enough that the command is still reasonably fast even with the additional time spent copying that file (especially considering how much longer "Refresh index ..." takes).

Spates

Answer 6

one way to go about this its to install "git for windows" package, it has the "git bash" tool that seems to be better integrated with windows and you don't have this "refreshing of the index" thing.

Yardmaster

Answer 7

For those who are still struggling with this issue and don't want to switch between two index files each time they open WSL, I found this interesting solution.

Create a bash script /usr/local/bin/git containing the following:

#!/bin/bash
if [[ "$(pwd -P)" =~ ^/mnt/./ ]]
    then exec /mnt/c/Program\ Files/Git/cmd/git.exe "$@"
    else exec /usr/bin/git "$@"
fi

This script will simply chose between git.exe or git according to the path you are on. As every directory that is in /mnt is usually related to one of your windows drive and you have probably used powershell git on them, this should work pretty well.

Source : https://gist.github.com/jasonboukheir/3fdab92ece236744528447a4f7f5de00

Thirteen