Git will not init/sync/update new submodules

Asked 26/7, 2010 at 16:47 Answered 28/10, 2022 at 9:1

203

Here's part of the contents of my .gitmodules file:

[submodule "src/static_management"]
        path = src/static_management
        url = git://github.com/eykd/django-static-management.git
[submodule "external/pyfacebook"]
        path = external/pyfacebook
        url = http://github.com/sciyoshi/pyfacebook.git

However, .git/config only contains the first:

[submodule "src/static_management"]
        url = git://github.com/eykd/django-static-management.git

The second submodule (external/pyfacebook) was added by another developer in a feature branch. I've inherited the development now, and have checked out the feature branch. However, Git will not pull the submodule for me. I've tried:

git submodule init
git submodule update
git submodule update --init
git submodule sync
Removing all submodule definitions from .git/config and running git submodule init. It only copies over the previously existing submodule and ignores the new one.
Entering new submodule definitions in .git/config manually and running git submodule update. Only the previously existing submodules bother to update.

in various combinations, but git simply will not update .git/config based on the new contents of .gitmodules, nor will it create the external/pyfacebook folder and pull the submodule's contents.

What am I missing? Is manual intervention (adding a submodule entry by hand to .git/config) truly required, and why?

Edit: Manual intervention does not work. Manually adding the new submodule entry to .git/config doesn't do a thing. The new submodule is ignored.

Gymnasiarch answered 26/7, 2010 at 16:47 Comment(2)

running 1.7.7.1 and having the same problem: "git submodule sync" does not update .git/config after a change to .gitmodules. – Instancy 6/12, 2011 at 12:13

This article is helpful: chrisjean.com/git-submodules-adding-using-removing-and-updating – Moluccas 28/7, 2015 at 19:46

170

I had this same problem - it turned out that the .gitmodules file was committed, but the actual submodule commit (i.e. the record of the submodule's commit ID) was not.

Adding it manually seemed to do the trick - e.g.:

git submodule add http://github.com/sciyoshi/pyfacebook.git external/pyfacebook

(Even without removing anything from .git/config or .gitmodules.)

Then commit it to record the ID properly.

Adding some further comments to this working answer: If the git submodule init or git submodule update does not work, then as described above git submodule add <url> should do the trick. One can cross check this by

 git config --list

and one should get an entry of the submodule you want to pull in the result of the git config --list command. If there is an entry of your submodule in the config result, then now the usual git submodule update --init should pull your submodule. To test this step, you can manually rename the submodule and then updating the submodule.

 mv yourmodulename yourmodulename-temp
 git submodule update --init

To find out if you have local changes in the submodule, it can be seen via git status -u ( if you want to see changes in the submodule ) or git status --ignore-submodules ( if you dont want to see the changes in the submodule ).

Backchat answered 28/1, 2012 at 23:58 Comment(7)

What is external/pyfacebook for? – Moluccas 28/7, 2015 at 19:41

@Moluccas That's the destination path for your submodule. – Esquibel 29/7, 2015 at 3:24

This helped me, thanks a lot! I could just add that if the destination path already exists (which it did for me as a result of trying other commands) one gets the following message which just adds to the confusion: 'your/local/path' already exists and is not a valid git repo – Crayton 24/2, 2016 at 14:6

one liner for readding entries in "git config --list":

git config --list | grep submodule | sed -e "s/submodule\.//" -e "s/\(.*\)\.url=\(.*\)/git submodule add --force \2 \1/" | bash

– Cuticle 16/5, 2018 at 9:43

I've experienced this issue a few times, and finally figured out why. Aside from cloning the submodule and updating the configuration files, git submodule add also stages a special non-file entry that marks the submodule's current commit ID. If you ever unstage this special entry, you lose it because it wasn't a file on your system to begin with. The submodule folder remains and its name still appears in the configuration files, but it just doesn't work. Running git submodule add again stages the special submodule commit ID entry again, which should then be committed to make it permanent. – Ott 24/9, 2022 at 6:14

@PugganSe Nice one. I made another one for executing the add command for the URLs listed in .gitmodules: https://mcmap.net/q/129526/-adding-git-submodules-automatically-gitmodules – Palatal 8/9, 2023 at 8:32

wow, such a design flaw of git. Is there a way to force-sync them? – Starbuck 20/10, 2023 at 3:25

115

git version 2.7.4. This command updates local code git submodule update --init --force --remote

Herwig answered 21/11, 2016 at 11:15 Comment(6)

Doing nothing for me. – Indicator 27/12, 2016 at 15:38

with regard to git-submodule [documentation)(git-scm.com/docs/git-submodule#git-submodule---remote) the aforementioned command should update local branch of the submodules. – Herwig 27/12, 2016 at 17:5

You can update individual module with git submodule update --init --force --remote <module-name>. – Norvin 17/6, 2019 at 22:16

does not work for me either. – Jacquline 3/1, 2023 at 0:55

@thomas The suggestion is that rather than simply downvoting, you should attempt to improve the answer with an edit. Perhaps it is for the best not to comment if you are unable to take criticism without making it personal. – Jeroboam 13/7, 2023 at 15:35

@thomas why would you have changes in a submodule? This works as expect – Ackerley 9/11, 2023 at 19:52

Had the same issue, when git ignored init and update commands, and does nothing.

HOW TO FIX

Your submodule folder should be committed into git repo
It shouldn't be in .gitignore

If that requirements met, it will work. Otherwise, all commands will execute without any messages and result.

If you did all that, and it still doesn't work:

Add submodule manually, e.g. git submodule add git@... path/to
git submodule init
git submodule update
commit and push all files - .gitmodules and your module folder (note, that content of folder will not commit)
drop your local git repo
clone a new one
ensure that .git/config doesn't have any submodules yet
Now, git submodule init - and you will see a message that module registered
git submodule update - will fetch module
Now look at .git/config and you will find registered submodule

Cercaria answered 27/10, 2015 at 13:45 Comment(3)

I believe the path to the submodules CAN be in .gitignore. At least I made it work by following the answer from @DaveJamesMiller. Nothing else worked for me. – Agamemnon 11/9, 2018 at 9:25

I had the submodule folder in .gitignore and no .gitmodules. First one was by accident, second one intended. So I did git submodule add .. after deleting the line in the ignore et voila. – Multicolored 6/6, 2021 at 18:30

I don't think that was a long or convoluted enough set of steps. Can we really claim that this is the Git way of doing things and it's how the functionality was intended?? – Navel 23/6, 2023 at 15:4

Thinking that manually setting up .gitmodules is enough is WRONG

My local git version 2.22.0 as of this writing.

So I came to this thread wondering why wasn't git submodule init working; I setup the .gitmodules file and proceeded to do git submodule init ...

IMPORTANT

git submodule add company/project.git includes/project is required (when adding the module for the first time), this will:
- add config to .git/config
- update the .gitmodules file
- track the submodule location (includes/project in this example).
you must then git commit after you have added the submodule, this will commit .gitmodules and the tracked submodule location.

When the project is cloned again, it will have the .gitmodules and the empty submodules directory (e.g. includes/project in this example). At this point .git/config does not have submodule config yet, until git submodule init is run, and remember this only works because .gitmodules AND includes/project are tracked in the main git repo.

Also for reference see:

Inutile answered 12/6, 2019 at 5:33 Comment(3)

Manually updating .git/config file as well as .gitmodule file did the trick. Thanks. – Isogamy 8/8, 2022 at 4:41

Both of editing .git/config and .gitmodule do not work. The key is after adding it into .gitmodule, you need to commit the changes for indexing by git add {your submodule folder} and git commit – Poirier 29/2 at 7:13

Thanks! I had to delete .gitmodules commit&push, then I ran git submodule add https://site folder/folder then myu submodule finally appeared! – Sensible 13/3 at 9:23

I had the same problem but none of the solutions above helped. The entries in the .gitmodules and in .git/config were right but the command git submodules update --init --recursive was doing nothing. I also removed the submodule directory and did run git submodules update --init --recursive and got the submodule directory back but with exactly the same commit as before.

I found the answer on this page. The command is:git submodule update --remote

Schizogenesis answered 21/6, 2019 at 13:44 Comment(1)

This was also the correct solution for me. I was running git submodule update instead of git submodule update --remote. – Ander 29/7, 2019 at 1:16

This means the submodules haven’t been set up correctly and a git submodule add command will have to be executed. A detailed explanation of how submodules work:

If submodules were not added correctly:

If not committed correctly, and the submodules and their corresponding folders are not already recorded in the index, then first every submodule in the .gitmodules file needs to be individually added via git submodule add before one can proceed.

Note: as mentioned here, Git does not currently have a command to do this at once for multiple submodules present in a .gitmodules file.

What does git submodule add do?

When a repo with submodules has been set up correctly and someone has already performed a git submodule add command, the command has done the following things:

create a folder for the submodule to reside in if it doesn’t already exist
clone the project as a submodule
set the submodule parameters in the shareable .gitmodules file
set the submodule parameters in your private .git/config file
create a .git folder for the submodule in your superproject’s private .git/modules folder
and also: record in your superproject’s index for each submodule in which folder it will reside, and in what state it should be (the hash commit code of the submodule.)

Point 3 and 6 are relevant when you clone the superproject, and point 6 indicates whether a git submodule add has been performed correctly and was committed. You can check point 6 by seeing whether the folders in which the submodules should reside are already there and empty (Note: this does not require a .keep or .gitignore mechanism to be in place, since this is part of a Git mechanism.) After cloning the superproject you can also perform a git submodule to see which submodules are expected.

You can then proceed by doing:

git submodule

will show the submodules present in the tree and their corresponding commit hash code, can serve as an initial check to see which submodules are expected
git submodule init

copies the submodules parameters from the repo’s .gitmodules file to your private .git/config file (point 4)
git submodule update

clones the submodules to a commit determined by the superproject and creates the submodules' .git folders under your superproject's .git/modules/ folder (points 2 and 5)
git submodule update --remote

same as update but sets the submodule to the latest commit on the branch available by the remote repo, similar as going in each submodule’s folder afterwards and do a git pull
or --->: git submodule update --init --remote

which is all of the above combined.

Alternatively, when the repo is set up correctly, you can also do a git clone with the --recursive flag to include the submodules with the init and update performed on them automatically.

Eisteddfod answered 23/11, 2020 at 17:11 Comment(2)

git submodule update --init --remote - worked for me, thanks – Uncivilized 13/9, 2022 at 5:38

even when added correctly this does work... seems like a 'dud' feature to me, with 'dud' logic and 'horrific' impementation. 'init' needs to be rewritten and given an award for 'worst feature ever' for a software product. – Pouliot 17/9, 2022 at 4:57

There seems to be a lot of confusion here (also) in the answers.

git submodule init is not intended to magically generate stuff in .git/config (from .gitmodules). It is intended to set up something in an entirely empty subdirectory after cloning the parent project, or pulling a commit that adds a previously non-existing submodule.

In other words, you follow a git clone of a project that has submodules (which you will know by the fact that the clone checked out a .gitmodules file) by a git submodule update --init --recursive.

You do not follow git submodule add ... with a git submodule init (or git submodule update --init), that isn't supposed to work. In fact, the add will already update the appropriate .git/config if things work.

EDIT

If a previously non-existing git submodule was added by someone else, and you do a git pull of that commit, then the directory of that submodule will be entirely empty (when you execute git submodule status the new submodule's hash should be visible but will have a - in front of it.) In this case you need to follow your git pull also with a git submodule update --init (plus --recursive when it's a submodule inside a submodule) in order to get the new, previously non-existing, submodule checked out; just like after an initial clone of a project with submodules (where obviously you didn't have those submodules before either).

Indicator answered 27/12, 2016 at 17:52 Comment(4)

That's interesting, because git help submodule says this about init: "init: Initialize the submodules recorded in the index (which were added and committed elsewhere) by copying submodule names and urls from .gitmodules to .git/config." So it sure sounds like it should do exactly what you say it doesn't do... ? Time for a update on the git documentation? – Luu 14/1, 2017 at 1:13

@brad I don't think I said that - but I added a clarification for that specific case. Thanks. – Indicator 30/5, 2017 at 15:32

@CarloWood any idea why the writers of git submodules decided that --init should be necessary to get new submodules (instead of grabbing them automatically on update)? It seem like updating your repository should grab everything necessary unless it would destroy data. With --init it forces you to know that new submodules might have been created, or just always issue a --init every time in which case, again, it would seem that it should be enabled by default. – Ama 1/11, 2017 at 17:50

@Ama Obviously I have no idea why the writers of git submodules decided anything, but my guess is that "update" is reserved for updating something that already exists, and "init" is used to create something (locally) new. Under the hood the two are probably significantly different enough to warrant a different command. – Indicator 3/11, 2017 at 17:8

The problem for me is that the repo's previous developer had committed the submodules/thing folder as just a regular folder, meaning when I tried to run git submodule add ..., it would fail with: 'submodules/thing' already exists in the index, yet trying to update the submodule would also fail because it saw that the path did not contain a submodule.

To fix, I had to delete the submodules/thing folder, commit the deletion, then run the git submodule add command to add it back correctly:

git submodule add --force --name thing https://github.com/person/thing.git submodules/thing

Sibilla answered 12/11, 2019 at 4:37 Comment(0)

According to the answer from Dave James Miller I can confirm that it worked for me. The important thing here was to commit the subprojects commit ID. Just to have the entry in .gitmodules was not enough.

Here is an appropriate commit:

https://github.com/dirkaholic/vagrant-php-dev-box/commit/d5f4c40bdbd80eefbb5ac6029823733f591435ae

Atrocious answered 15/7, 2012 at 15:3 Comment(0)

Sort of magically, but today I ran git submodule init followed by git submodule sync followed by git submodule update and it started pulling my submodules... Magic? Perhaps! This is truly one of the most annoying experiences with Git…

Scratch that. I actually got it working by doing git submodule update --init --recursive. Hope this helps.

PS: Make sure you are in the root git directory, not the submodule's.

Ambiversion answered 19/5, 2014 at 1:10 Comment(3)

Nah this does absolutely nothing for me. – Moluccas 29/3, 2015 at 21:18

@Moluccas I edited the answer above with what worked for me. Let me know if it works! – Ambiversion 31/3, 2015 at 14:47

I tried your new commands, but they didn't do anything either. – Moluccas 1/4, 2015 at 20:3

I had the same problem.

.gitmodules had the submodule, but after a git submodule init command it wasn't in .git/config.

Turns out the developer who added the submodule also added the submodule directory to the .gitignore file. That doesn't work.

Kathernkatheryn answered 28/1, 2013 at 3:44 Comment(0)

For the record:
I created the same issue by adding an empty repository as submodule. In this case, there was no reference hash available for the submodule, leading to the error described by the original poster.

Force-adding the repository after having committed to it solved the issue (as in Arvids post)
git submodule add --force [email protected] destination

Frictional answered 19/7, 2018 at 15:36 Comment(0)

I had the same problem today and figured out that because I typed git submodule init then I had those line in my .git/config:

[submodule]
   active = .

I removed that and typed:

git submodule update --init --remote

And everything was back to normal, my submodule updated in its subdirectory as usual.

Tortilla answered 11/5, 2017 at 16:54 Comment(0)

I had a similar problem with a submodule. It just didn't want to be cloned/pulled/updated/whatever.

When trying to re-add the submodule using git submodule add [email protected] destination I got the following output:

A git directory for 'destination' is found locally with remote(s):
  origin        [email protected]
If you want to reuse this local git directory instead of cloning again from
  [email protected]
use the '--force' option. If the local git directory is not the correct repo
or you are unsure what this means choose another name with the '--name' option.

So, I tried to enforce the add command:
git submodule add --force [email protected] destination

That worked in my case.

Foregone answered 31/5, 2017 at 12:18 Comment(0)

Below sync command resolved the issue :

git submodule sync

Acaudal answered 3/3, 2020 at 10:34 Comment(1)

it does nothing. – Retread 15/9, 2020 at 10:25

Same as you I found that git submodule sync does not do what you expect it to do. Only after doing an explicit git submodule add again does a submodule url change.

So, I put this script in ~/bin/git-submodule-sync.rb:

https://gist.github.com/frimik/5125436

And I also use the same logic on a few post-receive git deploy scripts.

All I need to do now is edit .gitmodules, then run this script and it finally works like I thought git submodule sync was supposed to.

Quitrent answered 9/3, 2013 at 19:36 Comment(1)

This seems to happen only on some repos... possibly due to some bug in Git. It hasn't happened to me on newly created repositories for a long while, but way back when, it used to happen all the time on certain repos... – Quitrent 14/1, 2015 at 14:14

When I saw this today, a developer had moved part of the tree into a new sub-directory and it looks as though his git client did not record the updated Subproject rules in the tree, instead they were just nuked, leaving .gitmodules referring both to stale locations and to subprojects which no longer existed in the current tree.

Adding the submodules back in, and comparing the commit shas of the submodule to those found in git show $breaking_commit_sha (search for lines matching regexp ^-Subproject) to adjust as needed fixed things.

Bugaboo answered 30/4, 2014 at 21:0 Comment(0)

Deleting submodule dir and its contents ("external/pyfacebook" folder) if it exists before git submodule add ... might fix problems.

Boldfaced answered 18/5, 2017 at 15:18 Comment(1)

This was the problem for me. Someone had committed the "submodule" folder as just a regular folder, meaning when I tried to run "git submodule add ...", it would fail with: "'vendor/mobx-state-tree' already exists in the index", yet trying to update the submodule would also fail because it saw that the path did not contain a submodule). To fix, I had to delete the folder, commit the deletion, then run the git add command to add it back correctly. – Sibilla 12/11, 2019 at 4:34

Just sharing what worked for me:

git clone --recurse-submodules <repository path>

This clones the remote repository already including the submodules. This means you won't need to run git submodule update or init after cloning.

Employer answered 20/1, 2020 at 19:26 Comment(0)

According to the book of git, if you want to "pull" the default branch of a submodule into the local repo that you currently working on, try git submodule update --remote.

To learn about how to track another branch, please carefully check the document mentioned above.

There is an easier way to do this as well, if you prefer to not manually fetch > and merge in the subdirectory. If you run git submodule update --remote, Git will go into your submodules and fetch and update for you.

$ git submodule update --remote DbConnector
remote: Counting objects: 4, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 4 (delta 2), reused 4 (delta 2)
Unpacking objects: 100% (4/4), done.
From https://github.com/chaconinc/DbConnector
   3f19983..d0354fc  master     -> origin/master
Submodule path 'DbConnector': checked out 'd0354fc054692d3906c85c3af05ddce39a1c0644'

Headsail answered 28/10, 2022 at 9:1 Comment(0)

I had the same problem and in my case the solution was very easy:

delete empty folder (name of submodule)
run git submodule update

Petrel answered 18/10, 2022 at 9:4 Comment(0)

-1

Remove the submodule from your .git/config
Run git submodule init command
Go to your submodule directory and run git pull origin master

It should works now

Veronique answered 27/12, 2018 at 8:38 Comment(0)

-1

Please check your submodules directory.

If there is only a .git file in it, then delete it.

Now execute git submodule update --remote --init

Retread answered 15/9, 2020 at 10:30 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags