git bare repositories, worktrees and tracking branches

Question

I'm working with a code base where I need to be working on several branches at once for different purposes. So I clone to a bare repository and then set up some worktrees:

git clone --bare ssh://[email protected]/project/repo repo.git
cd repo.git
git worktree add ../branch-1 branch-1
git worktree add ../branch-2 branch-2
... someone else creates branch-3 and pushes is ...
git fetch origin +refs/heads/*:refs/heads/* --prune
git worktree add ../branch-3 branch-3

Now the branch-3 worktree isn't set to track the remote tree and trying to make it do so, I get into a horrible mess.

$ cd ../branch-3
$ git branch -u origin/branch-3
error: the requested upstream branch 'origin/refs/heads/feature/SW-5884-move-database-container-to-alpine-base-2' does not exist
hint: ...<snip>
$ git fetch +refs/heads/*:refs/remotes/origin/* --prune
$ git branch -u origin/branch-3
fatal: Cannot setup tracking information; starting point 'origin/feature/SW-5884-move-database-container-to-alpine-base-2' is not a branch.

What's the right magic to get this to work?

Sibilant

It's not a good idea to add work-trees to --bare repositories. I would recommend that you leave the bare repository bare and make a non-bare clone in which to make work-trees. (In particular --bare changes the fetch refspec in such a way that any fetch might seriously mess with your in-progress work.) – Aurelioaurelius

Okay, thanks for the advice. I had assumed that this was how worktrees were supposed to be used; if the repository is non-bare then you end up having to create a dummy branch to sit the repository itself on, because it's not possible to have both the repository and a worktree on the same branch at the same time. Would you like to put this into an answer I can accept? – Sibilant

git config --add remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" fixes everything – Ladykiller

Answer 1

First, a side note: if you are going to use git worktree add for nontrival periods (more than two weeks at a time), be sure your Git is at least version 2.15.¹

For your particular purpose I'd recommend not using git clone --bare. Instead, use a regular clone, followed by the git worktree adds that you intend to do. You note in a comment that:

... you end up having to create a dummy branch to sit the repository itself on, because it's not possible to have both the repository and a worktree on the same branch at the same time.

There are several simple workarounds for this:

• Pick one of the N work-trees that you'll add, and use that as the branch in the main work-tree:

   git checkout -b branch-1 ssh://[email protected]/project/repo branch-1
  

The drawback is that you now have a special distinguished "main" branch that you cannot just remove at any time—all the other branches depend on it.

• Or, after the clone, use git checkout --detach in the work-tree, to get a detached HEAD on the default branch:

   git clone ssh://[email protected]/project/repo repo.git
   cd repo.git
   git checkout --detach
  

• The only drawback to this second approach is that the work-tree is full of files, probably a waste of space. There's a solution to that as well: create a blank commit using the empty tree, and check that out:

   git clone ssh://[email protected]/project/repo repo.git
   cd repo.git
   git checkout $(git commit-tree $(git hash-object -t tree /dev/null) < /dev/null)
  

OK, the last one is not exactly obvious. It really is pretty simple though. The git hash-object -t tree /dev/null produces the hash ID of the empty tree that already exists in every repository. The git commit-tree makes a commit to wrap that empty tree—there isn't one, so we must make one to check it out—and prints out the hash ID of this new commit, and the git checkout checks that out as a detached HEAD. The effect is to empty out our index and work-tree, so that the only thing in the repository work-tree is the .git directory. The empty commit we made is on no branch and has no parent commit (it's a lone root commit) that we will never push anywhere.

---

¹The reason for this is that Git 2.5, where git worktree first appeared, has a bug I consider very bad: git gc never scans the added work-trees' HEAD files, nor their index files. If the added work-trees are always on some branch and never have any git added but uncommitted work, this never causes any problems. If uncommitted work does not sit around for at least 14 days, the default prune protection time suffices to keep it from being destroyed. But if you git add some work or commit on a detached HEAD, go on holiday for a month or otherwise leave this added work-tree undisturbed, and then come back to it, and a git gc --auto ran after that two-week grace period ran out, the files you saved have been destroyed!

This bug was fixed in Git 2.15.

---

Why --bare goes wrong

The root of the problem here is that git clone --bare does two things:

• it makes a bare repository (core.bare set to true) that has no work-tree and does no initial git checkout; and
• it changes the default fetch refspec from +refs/heads/*:refs/remotes/origin/* to +refs/heads/*:refs/heads/*.

This second item means that there are no refs/remotes/origin/ names, as you discovered. That's not easily repaired because of the (mostly hidden / internal) concept of refmaps, which show up very briefly in the git fetch documentation (see link).

Worse, it means that refs/heads/* will be updated on every git fetch. There is a reason git worktree add refuses to create a second work-tree that refers to the same branch that is checked out in any existing work-tree, and that is that Git fundamentally assumes that no one will mess with the refs/heads/name reference that HEAD is attached-to in this work-tree. So even if you did work around the refmap issue, when you run git fetch and it updates—or even removes, due to --prune and an upstream removal of the same name—the refs/heads/name name, the added work-tree's HEAD breaks, and the work-tree itself becomes problematic. (See Why does Git allow pushing to a checked-out branch in an added worktree? How shall I recover? Note that this bug was fixed in Git 2.26, released in March 2020.)

There's one other thing you could try, which I have not tested at all, and that is: do the bare clone as you are doing, but then change the refspec and re-fetch, and delete all the existing branch names since they have no upstreams set (or, equivalently, set their upstreams):

git clone --bare ssh://[email protected]/project/repo repo.git
cd repo.git
git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'
git fetch
git for-each-ref --format='%(refname:short)' refs/heads | xargs git branch -d

(or replace the xargs with xargs -n1 -I{} git branch --set-upstream-to=origin/{} {}).

Aurelioaurelius

I like your third option but do you know the windows equivalent of that commit (using /dev/null)? I ran it with git bash but I am developing a build environment for a Windows developer and automating these commands using batch files. – Deputize

I don't know how you simulate /dev/null on Windows. Ancient DOS had NUL: like CON: and the like (I'm not sure if I am spelling these right), perhaps that works; in any case, I've been told that WIndows has something that does work. – Aurelioaurelius

I tried NUL. Didn't work. I can work around it. I was just surprised when I could not run the command substituting NUL. – Deputize

what about git clone --no-checkout ? – Sweitzer

@JohanBoulé: --no-checkout simply skips the final checkout step. However, in a non-bare repository, Git still creates an initial branch name (in some sense it shouldn't, but HEAD is still forced to hold a branch name, so it must). Since the index is literally empty at this point, all files are staged for deletion (!). It's not a great situation. – Aurelioaurelius

@Aurelioaurelius I tried the last suggestion you gave when cloning bare. It works as far as my initial testing shows. – Delamare

It looks like branches checked out in all worktrees are respected as of Git 2.26: github.com/git/git/commit/… – Vachon

@bb010g: Good find; I added a remark to the linked question as well. – Aurelioaurelius

For Windows the following worked for me: Although I am unable to encapsulate it into one single command, NUL works. git mktree <NUL gives you the hash of the empty tree. Copy it. git commit-tree YOUR_HASH_GOES_HERE <NUL will make that commit. It will output that commit hash - it is different from the tree hash. Copy it. git checkout THE_2ND_HASH_GOES_HERE will finally checkout the commit. – Layla

This works so well that now one of the first things I do when setting up a new development machine is to create ~/bin/git-make-bare which does your third option. – Sibilant

This has worked very well in my initial testing: git clone --bare --config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* clone-url local-path.git – Pontine

Answer 2

I thing there is a very simplest way;

1. clone the repo without checkout (no waste space)

git clone --no-checkout ssh://[email protected]/project/repo repo.git

2. go to the repo

cd repo.git

3. create a dummy branch to have the possibility to create a worktree on each existing one

git switch -c dummy

4. Now create your worktree as you want

git worktree add branch-1

or

git worktree add pathtobr1 branch-1

That's all. It is clean, easy without wasting space

Hope this helps ;-)

Links

Thanks, I wasn't aware of --no-checkout. Is it new? – Sibilant

Doesn't it work to just use git checkout $(git rev-parse HEAD) to have the clone use a detached HEAD, rather than creating a dummy branch? – Finfoot

With --no-checkout, HEAD does not exist. I don't like this situation because for some tools, a .git repository without HEAD is not recognised as repository. So, the dummy branch is a low price (actually, it is just put one text line in HEAD ;-) ). So, it is possible to work with one worktree per branch. – Links

Answer 3

I've struggled with this for a long time, never remembering the steps I took to create a bare repo that didn't act like one. Eventually I wrote the following script called git-clone-bare-for-worktrees to create bare repos for use with worktrees. It seems to work fairly well, but beware that it isn't doing a lot of error handling.

#! /bin/env bash
set -e

url=$1
name=${url##*/}

git init --bare "${name}"
cd "${name}"
git config remote.origin.url "$url"
git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'
git fetch

firstCommit=$(git rev-list --all --max-parents=0 --date-order --reverse | head -n1)
git branch bare-dummy $firstCommit
git symbolic-ref HEAD refs/heads/bare-dummy

It creates a branch named bare-dummy pointing to the first commit in the repo and sets that to HEAD, ensuring that all "real" branches are available to be safely checked out in worktrees. Other than that, the repo won't contain any local branches, not even master but remote tracking branches will have been created exactly as with a normal, non-bare clone. So just run a quick git worktree add ../master-worktree master and you should be up and running.

Eolanda

That's interesting and useful. Personally, I've created a git-make-bare script which does @torek's third bullet-point in the accepted answer. My workflow when cloning is then git clone <repo>; cd repo; git make-bare; git worktree add master master. – Sibilant

@tom, what do you do about HEAD in the bare repo? My technique of creating a dummy branch off of the first commit is pretty lame, but it's the best I've found so far to keep the bare repo from "hogging" a branch. – Eolanda

My git-make-bare basically does this: git checkout $(git commit-tree $(git hash-object -t tree /dev/null) < /dev/null) – Sibilant

@tom, I remember trying to use a detached HEAD in the bare repo, but if I recall correctly, some worktree operations complained if that was the case (even though I couldn't see why the operation needed a branch). That would have been with a much earlier version of git worktree though, so I should probably give that another try. – Eolanda

It works for me, but then I only use add and prune. – Sibilant

I found a slightly different approach and similar script that might be a useful: morgan.cugerone.com/blog/…. See also github.com/marakim/git-clone-for-worktree – Kiloton

Answer 4

Another solution would be to use the --separate-git-dir argument when cloning:

$ mkdir project && cd project

$ git clone http://****/wt.git main --separate-git-dir=.git
Cloning into 'main'...
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (6/6), done.

$ git worktree add wt1 b1
Preparing worktree (new branch 'b1')
branch 'b1' set up to track 'origin/b1'.
HEAD is now at 23b2efc Added file

$ ls -a
.  ..  .git  main  wt1

The main downside of this technique is that the root project directory behaves like an out-of-sync worktree if you attempt to run git commands there:

project$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        main/
        wt1/

no changes added to commit (use "git add" and/or "git commit -a")

But everything else looks to work exactly as intended.

To me this looks like the best compromise of all answers here.

Wilbert