How to pull a new submodule

Question

Tried looking for answers on this site and others: StackOverflow - Easy way pull latest of all submodules

They all seem to want to talk about if you are controlling them, not if someone else added one, and I just want to pull the additional one into my project without having to stash or backup my changes if I need to delete the folder.

Should I delete the .gitmodules file, and/or the submodule directories that I have already pulled down with git clone --recursive? (StackOverflow - How to git clone including submodules?)

These commands do not seem to help either:

• git submodule update --init --recursive seems like it did nothing.
• git submodule update --recursive nothing.
• git fetch --recurse-submodules output Fetching submodule ... several times.
• git pull --recurse-submodules output the same thing, and then said Already up-to-date. after the fetch trial. Strange since in either case my submodules were already downloaded.
• git clone --recursive ... Not tried yet. I feel like would overwrite any changes I have made, in the Stash or otherwise.
• git submodule update --recursive --remote checked out a new commit SHA for one of the submodules.
• git submodule update --recursive checked out a new commit SHA for one of the submodules. Could be the older, original commit level.
• git submodule status gives the appropriate SHA, version, and name information for each, while still lacking the one that I want.
• git submodule foreach git pull origin master
• git submodule update does nothing.

I have been double-checking the library directory manually each time to make sure whether the additional submodule appeared or not.

I want to avoid performing certain actions, unless they are not destructive to my current repository state containing code changes, and solves my problem, in case it is a command I have mentioned but did not run, or anyone else has another to try.

I could try some of these with more effort, but I think I want to stop messing with them for now, and since I have not found the answer to this issue after doing some online searching, maybe the hopeful and eventual answer would help others anyway.

Am I suffering from the con mentioned here at all? Software Engineering - Git submodule vs Git clone

More links:

• StackOverflow - Update Git submodule to latest commit on origin

Patchouli

Thanks for listing all the possibilities. git fetch --recurse-submodules did work for me. – Pulsometer

You knw what, I am reading your question and now I see it is contra-intuitive. Which why you may want to consider creating new issue ticket for git team. I guess pull must automatically process submodules appear/disappear. – Palsgrave

Answer 1

When you want to pull a specific new module, you can use the following command:

git submodule update --init local/path/to/submodule/folder

If you wish to initialize all new submodules however. you can omit the path to the folder like so:

git submodule update --init

Getz

Isn't there a way to run one command and get all new sub-module auto-initialized if still empty? – Operate

git submodule update --init (without specifying the folder path) seems to pull all new sub-modules – Alemannic

If there are some changes to other submodules (but commit sha not updated in parent repo), will the above command (git submodule update --init) remove all those changes and bring the submodule to original state? – Rancell

Answer 2

You have to do two things:

1. Do git pull in your main repository which holds the submodules. This will add the new submodule as an empty directory.
2. Do git submodule update --recursive --remote in the main repository. This will pull the latest changes for all submodules, including the new one.

This works at least in Git 2.13. Also note that if the repositories and submodules are on GitHub, you have to make sure you have access rights to them (if they are private).

Capo

Works well for me! Only note is that it doesn't only pull a new submodule, it updates all submodules. Also, I think for GitHub, you only need access rights if you use [email protected]:user/repo. If you use https://github.com/user/repo you don't need ssh access. – Reiss

@CodyReichert I tried that and it didn't do anything for me on Ubuntu 16.04. The folders of new submodules remain empty. I have git 2.7.4. Could it be that it was enhanced in newer versions? – Operate

Answer 3

Best suggestion I have received so far is to run this command:

git submodule add <URL_to_submodule> <local_path_to_place_submodule>

So it looks to be what the other contributor would have done, that I would do again, even though it already exists in the remote.

I guess this doesn't technically update the .gitmodules file from the remote's data like what would be expected, but haven't found a way to do that yet.

Credit for the help goes to @pandatrax.

Update

Before trying the add method, I tried 1 more idea that involved copying the .gitmodules file from the remote manually and trying any of the update commands, but sadly that approach did not work either. It may have gone differently if I executed the commands in the root, since I was in a subfolder, but I doubt it.

Then I used the add method, which downloaded the dependency, but the .gitmodules file showed changes. Once I set-up the GitHub remote and pulled from it after discarding that file, the project seems to be in a better state now, even syncing the commit SHAs for the updated module or 2, either since they matched, or it was overwritten.

Patchouli

Aaron's answer is much better than re-adding. You should accept his answer as the one. Although it doesn't work recursively, unfortunately. – Operate

no @AlexisWilke, he has point. if submodule update --init --recurse did not help then most likely this submodule itself was not comitted and needs to bew readded and commited – Palsgrave

Answer 4

This should work, as long as the new submodule is in the .gitmodules in remote

git pull --recurse-submodules

Minima

I tried that too, and it did nothing. The submodule directories remain empty. Could it be that if you do not include the --recurse-submodules the first time you do a git pull then it's in a state that prevents the recursion? – Operate

Answer 5

It's solved for my submodules when I updated submodule using the following command:

git pull <URL_to_submodule_or_repository> <Name_of_branch>

Kedgeree

Answer 6

not if someone else added one, and I just want to pull the additional one into my project without having to stash or backup my changes if I need to delete the folder.

This should work better with Git 2.36 (Q2 2022): when "git fetch --recurse-submodules"^(man) grabbed submodule commits that would be needed to recursively check out newly fetched commits in the superproject, it only paid attention to submodules that are in the current checkout of the superproject.

Git now does so for all submodules that have been run "git submodule init"^(man) on.

See commit 5fff35d, commit b90d9f7, commit 5370b91, commit 73bc90d, commit 6e1e0c9, commit 1e5dd3a, commit 7c2f8cc, commit d1d1572, commit 6e94bd6, commit f3117df (07 Mar 2022) by Glen Choo (chooglen).
^{(Merged by Junio C Hamano -- gitster -- in commit dd9ff30, 25 Mar 2022)}

fetch: fetch unpopulated, changed submodules

^{Signed-off-by: Glen Choo}

"git fetch --recurse-submodules"^(man) only considers populated submodules (i.e.
submodules that can be found by iterating the index), which makes "git fetch"^(man) behave differently based on which commit is checked out.
As a result, even if the user has initialized all submodules correctly, they may not fetch the necessary submodule commits, and commands like "git checkout --recurse-submodules"^(man) might fail.

Teach "git fetch to"^(man) fetch cloned, changed submodules regardless of whether they are populated.
This is in addition to the current behavior of fetching populated submodules (which is always attempted regardless of what was fetched in the superproject, or even if nothing was fetched in the superproject).

A submodule may be encountered multiple times (via the list of populated submodules or via the list of changed submodules).
When this happens, "git fetch" only reads the 'populated copy' and ignores the 'changed copy'.
Amend the verify_fetch_result() test helper so that we can assert on which 'copy' is being read.

fetch-options now includes in its man page:

submodules should be fetched too. When recursing through submodules, git fetch always attempts to fetch "changed" submodules, that is, a submodule that has commits that are referenced by a newly fetched superproject commit but are missing in the local submodule clone. A changed submodule can be fetched as long as it is present locally e.g. in $GIT_DIR/modules/ (see linkgit:gitsubmodules[7]); if the upstream adds a new submodule, that submodule cannot be fetched until it is cloned e.g. by git submodule update.

When set to 'on-demand', only changed submodules are fetched. When set to 'yes', all populated submodules are fetched and submodules that are both unpopulated and changed are fetched. When set to 'no', submodules are never fetched.

When unspecified, this uses the value of fetch.recurseSubmodules if it is set (see git config), defaulting to 'on-demand' if unset. When this option is used without any value, it defaults to 'yes'.

git fetch now includes in its man page:

Using --recurse-submodules can only fetch new commits in submodules that are present locally e.g. in $GIT_DIR/modules/. If the upstream adds a new submodule, that submodule cannot be fetched until it is cloned e.g. by git submodule update. This is expected to be fixed in a future Git version.

Interceptor