Can I store the .git folder outside the files I want tracked?

Question

I have an unusual idea to use git as a backup system. So let's say I have a directory ./backup/myfiles and I want to back that up using git. To keep things clean I don't want to have a .git directory in the myfiles folder, so I thought I could create ./backup/git_repos/myfiles. From looking at the git docs, I've tried doing this:

$ cd backup/myfiles
$ mkdir ../git_repos/myfiles
$ git --git-dir=../git_repos/myfiles init
Initialized empty Git repository in backup/git_repos/myfiles/
$ git --git-dir="../git_repos/myfiles/" add foo
fatal: pathspec 'foo' did not match any files

You can see the error message I get there. What am I doing wrong?

Viera

As well as your backup idea, this can also be used to keep your "dotfiles" (.bashrc, .vimrc, etc) in the home directory while keeping the .git folder elsewhere. – Industrial

Most straighforward answer: https://mcmap.net/q/12154/-can-i-store-the-git-folder-outside-the-files-i-want-tracked (buried because of old upvotes) – Eliathas

In the case that you have no write access or do not want to make any changes to the working directory (like adding .git/ etc.), this answer below by Leo (also buried by old upvotes) is the best. – Inward

@Philip, unless your dotfiles repository also contains Git submodules. Git doesn't support submodules in combination with an external work tree. – Mont

Answer 1

git --git-dir=../repo --work-tree=. add foo

This will do what you want but will obviously suck when you have to specify it with every git command you ever use.

You can export GIT_WORK_TREE=. and GIT_DIR=../backup and Git will pick them up on each command. That will only comfortably allow you to work in a single repository per shell, though.

I’d rather suggest symlinking the .git directory to somewhere else, or creating a symlink to the .git directory from your main backup directory.

Monas

You can archive the same without specifying the git-dir and the work-tree in every command and without any symbolic links. See my answer. – Foliate

The downside of a symlink is that it exists within the work tree, and if some other process wipes out the work tree, then you've lost the symlink – Brinkmanship

Furthermore, if the OP did not want a .git subdirectory in his work tree, why would he want a symlink? – Brinkmanship

@Jeff: for a trade-off. (In his backup case, wiping out his work tree is probably not any bigger concern for him than wiping out any other dir (like the repo itself).) – Piranha

I create scripts or aliases that look like – Zippel

I'm using this along with direnv for my notes. My working tree is in a folder in Dropbox and my git folder somewhere outside. This way I can have my changes on all computers easily but still be able to check what changed and only commit when something significant has changed. Thanks – Ultraviolet

Answer 2

You just need to ensure that the repository knows where the work tree is and vice versa.

To let the repository know where the work tree is, set the configuration value core.worktree. To let the work tree know where it's git directory is, add a file named .git (not a folder!) and add a line like

gitdir: /path/to/repo.git

Since git 1.7.5 the init command learned an extra option for this.

You can initialize a new separate repository with

git init --separate-git-dir /path/to/repo.git

This will initialize the git repository in the separate directory and add the .git file in the current directory, which is the working directory of the new repository.

Previously to 1.7.5 you had to use slightly different parameters and add the .git file yourself.

To initialize a separate repository the following command links the work-tree with the repository:

git --git-dir=/path/to/repo.git --work-tree=. init && echo "gitdir: /path/to/repo.git" > .git

Your current directory will be the working tree and git will use the repository at /path/to/repo.git. The init command will automatically set the core.worktree value as specified with the --git-dir parameter.

You could even add an alias for this:

[alias]
    initexternal = !"f() { git --work-tree=. --git-dir=\"$1\" init && echo \"gitdir: $1\" >> .git; }; f"

Use git version control on a read-only working directory

With the knowledge above, you can even set up git version control for an working directory without having write permissions. If you either use --git-dir on every git command or execute every command from within the repository (instead of the working directory), you can leave out the .git file and therefore do not need to create any files within the working directory. See also Leos answer

Foliate

You can do this to an existing repo too: move the .git folder to wherever you want it, add the .git file to point to it, and then you can just carry on using the repo as you normally would – Chaulmoogra

Note that you can only issue git commands from the root of your repo. Going into subfolders confuses it! (This happens whether or not the given value for gitdir is relative or absolute.) – Chaulmoogra

The fix for this is to specify 'core.worktree' in the actual git config file, ie the one in the folder you point to in .git. – Chaulmoogra

Yes, that is what I described in my second sentence of my answer. The configuration value core.worktree is of course stored in the config file of the .git folder, wherer the .git file points to. – Foliate

The alias may not work with git 1.7.x and earlier. Older git changes to your home directory when running an alias in a directory that is not evidently part of a repo (no .git subdir?). If the echo throws the .git file into your home, or barfs about .git being a directory, upgrade to 1.8.x. – Rozanneroze

This should be the accepted answer. Sheer git beauty. Very user friendly. – Histo

the description of .git used as a file is found at git-scm.com/docs/gitrepository-layout – Brinkmanship

Perfect! I'm in a situation where I have to use TFS but I want to be able to easily branch and merge locally and this was a great solution for preventing VS from switching my source control provider. – Thomey

I find when I used these instructions the repo getting made became a bare repository and it gives the error fatal: core.worktree and core.bare do not make sense. Seems like just changing the config so it is not bare resolves that. – Mental

To make it work my command was a little bit different: git init --separate-git-dir <repo_name> C:/Users/<user>/my_folder – Mystic

I tried today with 2.29.2 and it was unnecessary to move the existing .git directory before the command. The command moves the directory automatically in a split second, no copying involved, just a folder rename. This helped me reduce an 880 MB work directory to 220 MB (i.e. the local Git repo is 660 MB in size), which neatly fits into a 512 GB RAM disk and speeds up my builds by 20% without the risk of losing any local commits. My IDE (IntelliJ IDEA) also accepts it without any issues. – Nutcracker

I have been doing this for a couple of months now. Everything went well until I wanted to do a simple squash operation. git decided to do the rebase by checking out everything to the directory containing the supposedly bare repo (outside my working directory), which of course makes no sense. Consequently it lost all my ignored files (they did not exist in the outside directory). No data loss, per se, but a total and utter mess. Beware! – Photoelectrotype

Answer 3

The --separate-git-dir option for git init (and git clone) can be used to accomplish this on my version of git (1.7.11.3). The option separates the git repository from the work tree and creates a filesystem agnostic git symbolic link (in the form of a file named .git) in the root of the work tree. I think the result is identical to niks' answer.

git init --separate-git-dir path/to/repo.git path/to/worktree

Receivership

+1 Using a command line option for init itself looks clearer and cleaner – Trinh

in windows, repo.git is created with its hiden attribute set. I then change it manually. Do you happen to know if this is safe? – Colza

+1 Yes git leaned this command line option in version 1.7.5, which was not available back then (if I remember right). I've updated my answer to suggest to use this parameter. – Foliate

Answer 4

I find it simpler to reverse the --work-tree and --git-dir directories used in niks' answer:

$ cd read_only_repos
$ git --work-tree=/some/readonly/location/foo/ --git-dir=foo init
$ cd foo
$ git status
On branch master

Initial commit

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        .file_foo
        bar
        ...

This approach has two advantages:

• It removes the need to have any command-line options or .git files. You just operate normally from within the root of the repository.
• It allows you to version a file system even if you don't own it. Git will only write to the repository location.

The only caveat I have encountered is that instead of using a .gitignore file, you edit info/exclude.

You can then use the repository read_only_repos/foo as a remote in your own repositories even if the original files are not under version control.

Determinate

Well this is exactly the same command. The only difference that I see is that you swapped the order of the --work-tree and --git-dir arguments. And off course you do not create the .git file in the working directory as you do not have write access to it. Nevertheless, using version control for a directory without write access to it is a nice use case. :-) – Foliate

You got it. The key is creating the repo outside of the working directory. – Determinate

I like this - it allows me to use git on directories I'm sharing with Dropbox, without the other participants even knowing git is in use. – Babysitter

Answer 5

It's conventional to name a directory that is a git repository that has its working tree in an unusual place with a '.git' extension, much like a bare repository.

mkdir ../git_repos/myfiles.git

If you had provided the --work-tree option at init time then this would have automatically set up the core.worktree config variable that means that git will know where to find the working tree once you specify the git directory.

git --git-dir=../git_repos/myfiles.git --work-tree=. init

But you can set this variable after the fact as well.

git --git-dir=../git_repos/myfiles.git config core.worktree "$(pwd)"

Once you've done this, the add command should work as expected.

git --git-dir=../git_repos/myfiles.git add foo

Stoltzfus

I've found if you cd to ../git_repos/myfiles.git first, rather than being in the real work tree, 'git add foo' will just work, and you don't need to specify --git-dir all the time. – Ploy

It's true, but I think that most people tend to work in their working tree rather than their repositories. Of course, if you're using a detached working tree you're probably doing something 'special' and may well be using some macros to help. – Stoltzfus

Answer 6

Use git inside the repo:

cd ./backup/git_repos/myfiles
git init --bare
git config core.worktree ../myfiles
git config core.bare false

From now on, you can use git inside the ./backup/git_repos/myfiles directory, without setting any environment variables or additional parameters.

Mickimickie

This seems like the best answer, but I get the message warning: core.bare and core.worktree do not make sense does that mean it hasn't worked? – Crumble

I still think is works, but it complains about being bare when setting the worktree. That's why I'm setting core.bare to false afterward. – Mickimickie

Ah! That makes more sense now. Thanks for that. – Crumble

Answer 7

To clarify the difference between the two options below, you can tell your git repository to track files somewhere else (core.worktree) or you can put a breadcrumb in your main worktree that points to a git repository folder somewhere else (--separate-git-dir). The core.worktree option does not need a breadcrumb file and has been a feature of git since first release. The second option is newer and requires a breadcrumb file, but doesn't require altering the repository configuration file.

Option 1: core.worktree

Initialize a non-bare repository outside of the path you want to track and set core.worktree to the path you want to track. You can do this using terminal commands to set this value or directly edit the repository config file to add:

worktree = <path to files to backup>

Do not make the repository folder a subfolder of this path; that would be recursive. You could possibly try this and simply ignore the repository folder, but I think git won't allow this scenario.

In your case, you would go to backup/git_repos/ to run the init command and could use the --git-dir=./myfiles option to override the default repository folder name. The commands would look like this:

cd backup/git_repos 
git init --git-dir=./myfiles
git config core.worktree backup/myfiles

NOTE 1: This is not the same as the newer ADDITIONAL worktree functionality of git. This is your main working tree which is used by all non-bare repositories. Those additional worktrees use breadcrumbs to subfolders of the git repository folder, much like Option 2 below.

NOTE 2: I recently tested a great many git GUIs for windows and only Git Extensions supports using core.worktree to move the main working tree.

GUIs that do not support core.worktree

SourceTree, Fork, Tower, GitKraken, GitHub Desktop, GitAhead, SmartGit*, and Git-Cola. You will want to stick to the terminal when using core.worktree.

* SmartGit conflaits this feature with Option 2 and is asking for a .git file. This is not required for core.worktree.

Option 2: --separate-git-dir

Initialize a repository at the path you want to backup using --separate-git-dir=<path to hold repository data>. This will use the specified path to hold the repository data and create a .git file in the initialization location that contains a line like this:

gitdir: <path to hold repository data>

For you, the commands would look like this:

cd backup/myfiles
git init --separate-git-dir=backup/git_repos/myfiles/

And your .git file in backup/myfiles/ will contain gitdir: backup/git_repos/myfiles/

You now operate git treating the location of the .git file as if that was the repository location.

Literatim

in option 1 you use git init --git-dir= but I think it must be git init --separate-git-dir= instead. Tested with GIT 2.36. – Seasonseasonable

@NickSdot: No, option 1 does not use --separate-git-dir, that is option 2 and the entire point of there being two options. --git-dir is used to rename the .git folder to something else, which is a separate piece of what OP wants. – Literatim

Answer 8

You could create a "nodgit" script (No Dot GIT) with somet like

#!/bin/sh
gits=/usr/local/gits
    x=`pwd`
    testdir() {( cd $1; pwd; )}
    while [ "$x" != "/" ]; do
      y=`echo $x|sed -e "s/\//__/g"`
      if ([ -d "$gits/$y" ]); then
        export GIT_DIR="$gits/$y"
        export GIT_WORK_TREE="$x"
        if ([ "$1" = "nodinit" ]); then
          mkdir -p "$GIT_DIR"
          git init --bare; exit $?
        elif ([ "$1" = "shell" ]); then
          bash; exit $?
        else
          exec git "$@"
        fi
      fi
      x=`testdir "$x/.."`
    done

You can call nodgit in place of git and it will set variables as necessary by looking for a git repo. For example say you have a (bare) repo in /usr/local/gits/__home__foo_wibbles and you are in in /home/foo/wibbles/one then it will find the correct working directory (/home/foo/wibbles) and repo.

Oh you can also use "nodgit shell" to get a shell with the correct vars set so you can use plain old git commands.

Refund

Answer 9

Assuming your myfiles directories already exists and has some content, could you live with this:

cd ~/backup
git init
git add myfiles

The .git directory will be in backup, not in myfiles.

Mesdemoiselles

Although that would sorta fix what I want, I'd rather just store the myfiles folder under git, and nothing else. – Viera

You can keep a selection of the files you want git to track using the .gitignore file. If you'd add * and !myfiles to it, only that directory will be tracked. But if you want a separate repo for another directory, you would have a problem... – Mickimickie

Answer 10

I create scripts that look like

~/bin/git-slash:

#!/usr/bin/sh

export GIT_DIR=/home/Version-Control/cygwin-root.git/
export GIT_WORK_TREE=/

git --git-dir=$GIT_DIR --work-tree=$GIT_WORK_TREE "$@"

exit $?

It's redundant to use --git_dir=$GIT_DIR, but reminds me that I can also set environment variables outside the script.

The above example is for tracking local changes to cygwin system files.

Can make one such script for any major project that needs this - but / without /.git is my main use.

The above is small enough to make a shell alias or function, if you eliminate the redundancy.

If I do this often enough, I would revive the workspace to repository mapping of

"Boxes, Links, and Parallel Trees: Elements of a Configuration Management System", 
in Workshop Proceedings of the Software Management Conference. 1989.

whose closest modern counterpart is Perforce mappings or views, supporting partial checkouts as well as non-colocation of workspace and repo.

Zippel