What does "upload-pack: not our ref" mean, when fetching git refs via --tags?

Question

In one of my projects, the Travis builds are failing before any of my build-system or code can be reached, as soon as my build-script attempts to fetch all of the Git tags with git fetch --tags:

`` git fetch --tags --verbose
POST git-upload-pack (350 bytes)
POST git-upload-pack (788 bytes)
POST git-upload-pack (797 bytes)
From https://github.com/ELLIOTTCABLE/bs-sedlex
 = [up to date]      fix-ci        -> origin/fix-ci
 * [new tag]         sedlex-1.99.2 -> sedlex-1.99.2
 * [new tag]         v1.99.3       -> v1.99.3
...
 * [new tag]         v20.0.0-pre.2 -> v20.0.0-pre.2
Fetching submodule ppx-sedlex
POST git-upload-pack (122 bytes)
From https://github.com/ELLIOTTCABLE/ppx-sedlex
 = [up to date]      develop       -> origin/develop
 = [up to date]      master        -> origin/master
...
 = [up to date]      v20.0.0-pre.2 -> v20.0.0-pre.2
POST git-upload-pack (4 bytes)
POST git-upload-pack (69 bytes)
POST git-upload-pack (586 bytes)
fatal: remote error: upload-pack: not our ref 0f509703fcd43ff4324d721a39220153bab49d4a

This is especially confusing, as neither the main repo bs-sedlex, nor the git-submodule ppx-sedlex, have any commit starting like 0f5097...; I have no idea where that SHA is coming from. This failure is occurring only on the Linux workers, and I can't figure out why — git fetch --tags on that same repo works on the macOS Travis-workers, on my macOS machine, and on an Ubuntu Vagrant box I spun-up to debug this.

What does the "fatal: remote error: upload-pack: not our ref" error signify; and how can I work around it? I'm not even sure where to begin debugging this error, as it only occurs specifically on Travis workers.

(It's unlikely to be helpful, but here's the error in context, and the repository in question.)

Edit 1: Here's some additional interesting output, from adding GIT_TRACE=2:

Fetching submodule ppx-sedlex
23:55:28.125076 git.c:439               trace: built-in: git fetch --no-prune --no-prune-tags --tags -v --recurse-submodules-default on-demand --submodule-prefix ppx-sedlex/
23:55:28.125914 run-command.c:663       trace: run_command: git-remote-https origin https://github.com/ELLIOTTCABLE/ppx-sedlex.git
23:55:28.429609 run-command.c:663       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
23:55:28.432485 run-command.c:663       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
23:55:28.434082 git.c:439               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs
From https://github.com/ELLIOTTCABLE/ppx-sedlex
 = [up to date]      develop       -> origin/develop
 = [up to date]      master        -> origin/master
 = [up to date]      v1.99.4       -> v1.99.4
 = [up to date]      v1.99.4-pre.1 -> v1.99.4-pre.1
 = [up to date]      v1.99.4-pre.3 -> v1.99.4-pre.3
 = [up to date]      v1.99.4-pre.8 -> v1.99.4-pre.8
 = [up to date]      v2.0.0        -> v2.0.0
 = [up to date]      v20.0.0-pre.1 -> v20.0.0-pre.1
 = [up to date]      v20.0.0-pre.2 -> v20.0.0-pre.2
23:55:28.442482 run-command.c:1616      run_processes_parallel: preparing to run up to 1 tasks
23:55:28.442504 run-command.c:1648      run_processes_parallel: done
23:55:28.442536 run-command.c:663       trace: run_command: git gc --auto
23:55:28.443983 git.c:439               trace: built-in: git gc --auto
23:55:28.444903 run-command.c:663       trace: run_command: cd /home/vagrant/ELLIOTTCABLE/bs-sedlex/.git/modules/ppx-sedlex; unset GIT_PREFIX; GIT_DIR=. git fetch --no-prune --no-prune-tags --tags -v --recurse-submodules-default on-demand --submodule-prefix ppx-sedlex/ origin 0f509703fcd43ff4324d721a39220153bab49d4a
23:55:28.446392 git.c:439               trace: built-in: git fetch --no-prune --no-prune-tags --tags -v --recurse-submodules-default on-demand --submodule-prefix ppx-sedlex/ origin 0f509703fcd43ff4324d721a39220153bab49d4a
23:55:28.447105 run-command.c:663       trace: run_command: git-remote-https origin https://github.com/ELLIOTTCABLE/ppx-sedlex.git
23:55:28.735871 run-command.c:663       trace: run_command: git fetch-pack --stateless-rpc --stdin --lock-pack --thin --no-progress https://github.com/ELLIOTTCABLE/ppx-sedlex.git/
23:55:28.738885 git.c:439               trace: built-in: git fetch-pack --stateless-rpc --stdin --lock-pack --thin --no-progress https://github.com/ELLIOTTCABLE/ppx-sedlex.git/
error: Server does not allow request for unadvertised object 0f509703fcd43ff4324d721a39220153bab49d4a

I can't make hide-nor-hair of why Git is requesting an "unadvertised object" here; but it's clearly not a GitHub problem, here — for some reason, the command:

git fetch --no-prune --no-prune-tags --tags -v \
   --recurse-submodules-default on-demand \ 
   --submodule-prefix ppx-sedlex/ \
   origin 0f509703fcd43ff4324d721a39220153bab49d4a

... is being automatically invoked upon the submodule, when I git fetch in the parent repo. (Again, that commit, 0f509703, does not exist in either repo; again, the exact same repo, the exact same commit, and this isn't happening on macOS — only on Travis's Linux machines.)

Heidi

working tree structure has damaged. Remove wrong refs in the path: my-project/.git/refs/heads – Strati

Answer 1

I had the same error:

radato$ git submodule update 
fatal: git upload-pack: not our ref somehash0
fatal: remote error: upload-pack: not our ref somehash0
fatal: Fetched in submodule path 'dashboard', but it did not contain somehash0. Direct fetching of that commit failed.

So I deinitialized the submodule:

radato$ git submodule deinit -f dashboard
Cleared directory 'dashboard'
Submodule 'dashboard' (ssh://[email protected]/somedomain/dashboard.git) unregistered for path 'dashboard'

Now checking the summary I get

radato@$ git submodule summary 
* somepath1 somehash1 (2):
  < Merged in fixed_edge (pull request #2)
  < fixed broken test

* somepath2 somehash2 (2):
  < Merged in dsp_tx_freq (pull request #14)
  < Merged in v0.2.0 (pull request #13)

Eventually I pulled:

radato@$ git pull --prune --recurse-submodules --force 
Fetching submodule dashboard
Fetching submodule somepath1
Fetching submodule somepath2
Already up to date.
Submodule path 'somesubmodulepath1': checked out 'somehash1'
Submodule path 'somesubmodulepath2': checked out 'somehash2'

And now it works again. Here is the status

radato$ git status 
On branch develop
Your branch is up to date with 'origin/develop'.

nothing to commit, working tree clean

Cecilacecile

Answer 2

This is especially confusing, as neither the main repo bs-sedlex, nor the git-submodule ppx-sedlex, have any commit starting like 0f5097...;

But they might have a tag with that SHA1 (which, once dereferenced, would point to a commit)

What does the "fatal: remote error: upload-pack: not our ref" error signify;

See "cloning a repo with nested submodules does not work"

Git provides three options which control whether you can fetch an arbitrary object ID:

• one which allows fetching any arbitrary object that Git has access to,
• one which allows fetching any object reachable from a reference,
• and one which additionally allows fetching objects reachable from hidden references.

The "not our ref" message means that you're trying to fetch an object by object ID, which is used for submodules, but the server doesn't allow it.

In your case, it might be possible that:

• either the tag in the submodule was never pushed
• or (since it is working from other sources) Travis-CI does not have access to the submodule (private dependencies): see "Git - Submodules in Travis CI ".
  Or it has some cached version of that submodule.

Karlene

Answer 3

Old question, but since there is no definite answer: This happens if a reference is not available on the remote, which can happen for several reasons:

• It belonged to a tag that has been deleted
• It belonged to a branch that has been removed
• The repository was forced pushed and the reference is not part of the new repodata
• There is some kind of access restriction that makes the reference private and not accessible to you
• properly a few more reasons

In any case, you should be able to get out of this mess, by removing your own stale metadata and fetching a new tree from git as described in the other answers.

Sporting

Answer 4

Possible diagnosis: Wrong upstream remote url

I had this error while trying to update submodules, and the issue was that the submodule remote was incorrect.

To see if this is your issue:

1. Check the remote url in the .gitmodules file for the submodule that's giving you problems.
2. Now cd into the into the submodule and run git remote get-url origin

Those URLs should match. If they don't match, then you/your organization probably forked a submodule from elsewhere and updated it, and you need to manually update the upstream remote url locally.

Solution

The .gitmodules file is probably correct, as whoever forked the submodule should have updated the .gitmodules file to point to the correct submodule.

Therefore:

1. Copy the remote URL from .gitmodules
2. cd back into the submodule directory
3. Run git remote set-url origin PASTE_URL_HERE
4. Check git remote get-url origin and confirm it now matches the .gitmodules file
5. Rerun git submodule update --init --recursive at the root of your project

Hope it helps someone!

Fimbriate

Answer 5

In my case I solved the problem by doing git reset --hard <for sure more than 100 commits before ref>.

Then I simply did git pull and it worked. Since it worked right away with the procedure I didn't try any other, so I can't provide a deterministic approach to fix this.

Nepheline

Answer 6

Ran into this on Arch PKGBUILD and a search identified this command git submodule sync for kame-tools-git

I discovered a similar problem with PKGBUILD for PPSSPP and adding git submodule sync fixed it. Don't know why, though.

==> Starting prepare()...
fatal: git upload-pack: not our ref 4524b3a324ae8e9dcaf80c4fe3694bc63628de22
fatal: remote error: upload-pack: not our ref 4524b3a324ae8e9dcaf80c4fe3694bc63628de22
fatal: Fetched in submodule path 'buildtools', but it did not contain 
4524b3a324ae8e9dcaf80c4fe3694bc63628de22. Direct fetching of that commit failed.

The pkg maintainer's fix is shown in this diff:

diff --git a/PKGBUILD b/PKGBUILD
index 3d3ea9ed024..37d01b46467 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -1,7 +1,7 @@
 # Maintainer: beelzy
 pkgname=kame-tools-git
 pkgver=5ae1ae1
-pkgrel=2
+pkgrel=3
 pkgdesc="A tool for bundling 3DS theme files"
 arch=('i686' 'x86_64' 'aarch64')
 url="https://gitlab.com/beelzy/kame-tools"
@@ -12,8 +12,8 @@ depends=('gcc-libs')
 _gitroot='https://gitlab.com/beelzy/kame-tools.git'
 _gitname='kame-tools'
 
-source=("${_gitname}::git+${_gitroot}"
-        "buildtools::git+https://github.com/Steveice10/buildtools.git")
+source=("git+${_gitroot}"
+        "git+https://github.com/Steveice10/buildtools.git")
 sha256sums=('SKIP' 'SKIP')
 
 pkgver() {
@@ -24,7 +24,9 @@ pkgver() {
 prepare() {
     cd "$srcdir/$_gitname"
     git submodule init
-    git submodule--helper update
+    git config submodule.buildtools.url $srcdir/buildtools
+    git submodule sync
+    git -c protocol.file.allow=always submodule update
     cd buildtools
     git apply ../aarch64.patch
 }

Tammara

Answer 7

https://developer.matomo.org/guides/git "Fixing the reference error When you forgot to push the submodule Let's say you make some changes in the submodule. Then you commit these changes, but you forget to push the changes like below:

cd plugins/SecurityInfo.
git checkout -b $MY_FEATURE_BRANCH $MAIN_BRANCH.
git add Controller.php && git commit -m "Update".
Here usually a push should happen, but you forgot it.
cd ../..
git add plugins/SecurityInfo this stages the submodule reference change
git commit -m 'Update submodule'
git push
When you push now, then the updating of the submodule won't work because the commit only exists on your local computer but was never pushed. "

To resolve this: You could pull a commit that does exist and update your main branch with the old commit. https://unfuddle.com/stack/tips-tricks/git-pull-specific-commit/

Mockheroic

Answer 8

My way to fix this was to:

1. Create a new branch in submodule, out of develop branch.
2. Then git cherry-pick <problematic commit/s hash>.
3. Then I just pushed that branch to remote develop and it got fixed.

I ran my build on Netlify, so I also had to restart the build, with cache clearing (that last thing is very important, as Netlify might try to get the old git history without it).

Hope that helps

Numbersnumbfish

Answer 9

I don't know the Travis case, but in my case in GITLAB I had similar issue when fetch project on my local machine. The issue was the tree on remote repository where in the refs path: my-project/.git/refs/heads there were not my branches, but a wrong directory including wrong file named as a previous branch containing 000000000000000 string, whereas I am supposed to see a sha1 string.

First >> run git -C my-project/.git fsck --no-dangling to see is there any error

Second >> in directory my-project/.git/refs/heads delete every directory or files NOT related to your familiar branches

Third >> in GITLAB case, I had to remove such wrong ref files in /var/opt/gitlab/git-data/repositories/@hashed/XX/YY/<hash-string-of-project>.git/refs/heads

Strati

Answer 10

try fix ***.git dir

git fsck --full
git gc --prune=now --aggressive

Copywriter