Git submodule fails with Could not access submodule

Question

Operating system: Linux

git version: 2.26.2

Git repo provider of my repo: gitlab

Repo provider of the failing submodules: Github

.gitmodules

[submodule "libraries/stb"]
    path = libraries/stb
    url = https://github.com/nothings/stb.git
    branch = master
[submodule "libraries/harfbuzz"]
    path = libraries/harfbuzz
    url = https://github.com/harfbuzz/harfbuzz.git
    branch = master
[submodule "libraries/shaderc"]
    path = libraries/shaderc
    url = https://github.com/google/shaderc.git
    branch = master
[submodule "libraries/freetype2"]
    path = libraries/freetype2
    url = https://github.com/aseprite/freetype2.git
    branch = master
[submodule "libraries/VulkanMemoryAllocator"]
    path = libraries/VulkanMemoryAllocator
    url = https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git
    branch = master
[submodule "libraries/googletest"]
    path = libraries/googletest
    url = https://github.com/google/googletest.git
    branch = master
[submodule "libraries/Eigen"]
    path = libraries/Eigen
    url = https://gitlab.com/libeigen/eigen.git
    branch = master
[submodule "libraries/benchmark"]
    path = libraries/benchmark
    url = https://github.com/google/benchmark.git
[submodule "libraries/rapidcheck"]
    path = libraries/rapidcheck
    url = https://github.com/emil-e/rapidcheck.git
    branch = master
[submodule "libraries/magic_get"]
    path = libraries/magic_get
    url = https://github.com/apolukhin/magic_get.git
    branch = master

command and output:

git pull --recurse-submodules=true

Fetching submodule libraries/Eigen
Fetching submodule libraries/VulkanMemoryAllocator
Fetching submodule libraries/benchmark
Fetching submodule libraries/freetype2
Fetching submodule libraries/googletest
Fetching submodule libraries/harfbuzz
Could not access submodule 'libraries/magic_get'
Could not access submodule 'libraries/rapidcheck'
Could not access submodule 'libraries/shaderc'
Could not access submodule 'libraries/stb'

git submodule status:

git submodule status 
 a145e4adf5e71391d64c0ab150a8c26851cf332d libraries/Eigen (before-git-migration-159-ga145e4adf)
 755fd47121ce0c77ef11818e4987790ae99c2eba libraries/VulkanMemoryAllocator (v2.1.0-334-g755fd47)
 d3ad0b9d11c190cb58de5fb17c3555def61fdc96 libraries/benchmark (v1.5.0-56-gd3ad0b9)
 fbbcf50367403a6316a013b51690071198962920 libraries/freetype2 (VER-2-10-0)
 dcc92d0ab6c4ce022162a23566d44f673251eee4 libraries/googletest (release-1.8.0-2331-gdcc92d0a)
 89ad3c6cc520517af15174391a9725e634929107 libraries/harfbuzz (2.6.5-71-g89ad3c6c)

git tree

git ls-tree -r HEAD
160000 commit a145e4adf5e71391d64c0ab150a8c26851cf332d  Eigen
160000 commit 755fd47121ce0c77ef11818e4987790ae99c2eba  VulkanMemoryAllocator
160000 commit d3ad0b9d11c190cb58de5fb17c3555def61fdc96  benchmark
160000 commit fbbcf50367403a6316a013b51690071198962920  freetype2
160000 commit dcc92d0ab6c4ce022162a23566d44f673251eee4  googletest
160000 commit 89ad3c6cc520517af15174391a9725e634929107  harfbuzz
160000 commit 762a4ec1f4458a84bc19cd6efc1e993add90ec95  libigl
160000 commit 03afd0b83ce67062a105cfcbe80bbca152743f0a  magic_get
160000 commit 7bc7d302191a4f3d0bf005692677126136e02f60  rapidcheck
160000 commit 011139094ec790ff7f32ea2d80286255fc9ed18b  shaderc
160000 commit b42009b3b9d4ca35bc703f5310eedc74f584be58  stb

This used to work about 6 months ago

Alixaliza

Has anything changed with the repos that you failed to fetch? Maybe their URL? Maybe the branch master doesn't exist anymore? – Suave

Not to my knowledge. All of those are fairly popular libraries and maintained by large organizations. I would be shocked if they were to break dependencies nily willy – Alixaliza

Answer 1

Just in case, make sure all submodules are initialized:

git submodule update --init --recursive

Then, try the pull --recurse-submodules=true again.

Somehow, activating the traces improve the situation:

git -c trace2.eventTarget=1 pull --recurse-submodules=true

Secretarial

Error persists after trying this method – Alixaliza

@Alixaliza Can you check which SHA1 is referenced by the missing submodules (git submodule status or see https://mcmap.net/q/193580/-how-to-see-which-commit-a-git-submodule-points-at/6309)? The goal is to check if that reference does exist in the remote repository. – Secretarial

Ran the command, the requested submodules don;t seem to show up on the list – Alixaliza

@Alixaliza And cd libraries plus git ls-tree -r HEAD? Would you see any entry for those submodules? – Secretarial

They did appear when I rant that command under libraries – Alixaliza

@Alixaliza OK. Strange. for example: github.com/apolukhin/magic_get/commit/… does exist, which means the repo does record an existing SHA. – Secretarial

@Alixaliza Activating trace2 (https://mcmap.net/q/12679/-how-can-i-debug-git-git-shell-related-problems, git-scm.com/docs/git-config#Documentation/…) should display more clues: git -c trace2.eventTarget=1 pull --recurse-submodules=true – Secretarial

@Alixaliza Also, for testing, try SSH instead of HTTPS for those GitHub URL: https://mcmap.net/q/673097/-how-to-use-ssh-instead-of-http-for-git-submodules git config "[email protected]:.insteadOf" https://github.com/ (assuming you have a working SSH key, already registered to your GitHub account) – Secretarial

The command seems to have worked, it pulled all the submodules i.e. the one with event target – Alixaliza

@Alixaliza The git -c trace2.eventTarget=1 pull --recurse-submodules=true? – Secretarial

@Alixaliza Strange! I have edited the answer accordingly. – Secretarial

I came here with the same problem. "git -c trace2.eventTarget=1 pull" removes the "Could not access submodule" error when trying to git pull the following repository: github.com/google/skywater-pdk - cloning is done from a Makefile in github.com/efabless/caravel-lite/blob/main/Makefile ("make pdk") which does not initialize all submodules on purpose, as not all are actually needed. This leads to the said error with git 2.25.1, though when not using "-c trace2.eventTarget=1". – Brouhaha

@Brouhaha Interesting. I wonder if the issue would persist with a more recent Git (2.34.1). – Secretarial

I had the same issue (Git 2.37) and I needed to remove the subdirectory first before the --init worked again. – Criminology

@RayOei Good point. If that subdirectory existed before the first update, it could indeed prevent the submodule initialization. – Secretarial