Issue with adding common code as git submodule: "already exists in the index"

Question

I want to add some git submodules. I've received two projects sharing some common code. The shared code was just copied into the two projects. I created a separate git repo for the common code and removed it from the projects with the plan to add it as a git submodule.

I used the path option of git submodule add to specify the folder:

git submodule add url_to_repo projectfolder

but then got the error:

'projectfolder' already exists in the index"

This is the desired structure of my repository:

repo
|-- projectfolder
    |-- folder with common code

It is possible to add the git submodule directly in the repo, or into a new folder there, but not in the project folder. The problem is that it really need to be in the project folder.. What can I do about this and what have I misunderstood about the path option of git submodule add?

Phemia

What do you get if you do git ls-files --stage projectfolder? – Downing

I get a list with all content starting with 100644. – Phemia

related: #12697419 – Fraktur

for me doing a git rm on the existing folder helped :| – Bainite

Answer 1

I'm afraid there's not enough information in your question to be certain about what's going on, since you haven't replied to my follow-up question, but this may be of help in any case.

That error means that projectfolder is already staged ("already exists in the index"). To find out what's going on here, try to list everything in the index under that folder with:

git ls-files --stage projectfolder

The first column of that output will tell you what type of object is in the index at projectfolder. (These look like Unix filemodes, but have special meanings in git.)

I suspect that you will see something like:

160000 d00cf29f23627fc54eb992dde6a79112677cd86c 0   projectfolder

(i.e. a line beginning with 160000), in which case the repository in projectfolder has already been added as a "gitlink". If it doesn't appear in the output of git submodule, and you want to re-add it as a submodule, you can do:

git rm --cached projectfolder

... to unstage it, and then:

git submodule add url_to_repo projectfolder

... to add the repository as a submodule.

However, it's also possible that you will see many blobs listed (with file modes 100644 and 100755), which would suggest to me that you didn't properly unstage the files in projectfolder before copying the new repository into place. If that's the case, you can do the following to unstage all of those files:

git rm -r --cached projectfolder

... and then add the submodule with:

git submodule add url_to_repo projectfolder

Downing

Thank Mark for your detailed answer. I did a "git rm -r --cached projectfolder" and tried to att the submodule again. However this time I get the error rmessage "'projectfolder' already exists and is not a valid git repo". – Phemia

@Vanja: That indicates that projectfolder didn't contain a .git directory. It sounded from your question as if you'd created the new repository for projectfolder elsewhere, and copied it into place, but clearly that wasn't the case. You'd need to move the existing projectfolder out of the way, and then copy the new repository (complete with its .git directory) into place before adding as a submodule. Or, if you've already pushed it to the repository represented by url_to_repo, you could just move projectfolder out of the way and then add the submodule from that URL. – Downing

This is what I did, which is 3 step process STEP 1: git rm -r --cached src/test/resources/ , STEP 2: removed existing resources directory to some other place , STEP 3: git submodule add add url_to_repo src/test/resources – Vouchsafe

Thanks, this helped me out when I had tried to add another repo in a hurry and did it the wrong way (git clone inside a repo, instead of git submodule add). On my other machine, it appeared to be a submodule, but no .gitmodules file was ever created. The above steps set me on the path to fixing the issue. Much obliged! – Robbyrobbyn

projectfolder shouldn't be a part of the existing parent repo, it would be created while running git submodule add. – Banas

Answer 2

You need to remove your submodule git repository (projectfolder in this case) first for git path.

rm -rf projectfolder

git rm -r projectfolder

and then add submodule

git submodule add <git_submodule_repository> projectfolder

Titanesque

git submodule add --force <git_submodule_repository> worked for me to keep the same submodule name – Spinneret

Thanks!! I didn't have to add in the project folder, I just changed into the subdirectory and clones from there. – Caloric

You'r a genius ! – Dimorph

Simple but its works. Thanks – Oculomotor

Don't forget to push submodule before rm command! – Forge

Answer 3

Removing submodule manually involves number of steps and this worked for me.

Assuming you are in the project root directory and sample git module name is "c3-pro-ios-framework"

Remove the files associated to the submodule

rm -rf .git/modules/c3-pro-ios-framework/

Remove any references to submodule in config

vim .git/config

Remove .gitmodules

rm -rf .gitmodules

Remove it from the cache without the "git"

git rm --cached c3-pro-ios-framework

Add submodule

git submodule add https://github.com/chb/c3-pro-ios-framework.git

Kory

Answer 4

I had the same problem and after hours of looking found the answer.

The error I was getting was a little different: <path> already exists and is not a valid git repo (and added here for SEO value)

The solution is to NOT create the directory that will house the submodule. The directory will be created as part of the git submodule add command.

Also, the argument is expected to be relative to the parent-repo root, not your working directory, so watch out for that.

Solution for the example above:

1. It IS okay to have your parent repo already cloned.
2. Make sure the common_code directory does not exist.
3. cd Repo
4. git submodule add git://url_to_repo projectfolder/common_code/ (Note the required trailing slash.)
5. Sanity restored.

I hope this helps someone, as there is very little information to be found elsewhere about this.

Cephalonia

Step 4 should begin git submodule add, and the trailing slash isn't required. – Backstay

Fixed the typo. I have retested it, and the trailing slash has made a difference for me. I'm using zsh if that has any effect. – Cephalonia

This solved my problem (creating the directory with the add submodule commnand), thank you! – Waggish

Thank you so so so much, this was my problem ... so relative path from inside your module to a folder including trailing slash !!! This solved it, thx. so for me the command looked like this for gitbash: git submodule add -b master "[email protected]:remote_repo.git" "mytools/src/project/grpc/protos/" – Hoist

"Sanity restored." Amen – Scepter

Answer 5

if there exists a folder named x under git control, you want add a same name submodule , you should delete folder x and commit it first.

Updated by @ujjwal-singh:

Committing is not needed, staging suffices.. git add / git rm -r

Cahilly

The committing was what I missed. Short answer, but correct one! – Cupidity

Committing is not needed, staging suffices.. git add / gir rm -r – Necessarian

Thanks. I moved my existing folder (an unrecognised subrepository) out of the master repo. I committed all changes to the master repo. Then I moved the subrepository back into it and was able to add it after that, using SourceTree. – Dimond

Answer 6

Just to clarify in human language what the error is trying to tell you:

You cannot create a repository in this folder which already tracked in the main repository.

For example: You have a theme folder called AwesomeTheme that's a dedicated repository, you try do dump it directly into your main repository like git submodule add sites/themes and you get this "AwesomeTheme" index already exists.

You just need to make sure there isn't already a sites/themes/AwesomeTheme in the main repository's version tracking so the submodule can be created there.

So to fix, in your main repository if you have an empty sites/theme/AwesomeTheme directory then remove it. If you have already made commits with the sites/theme/AwesomeTheme directory in your main repository you need to purge all history of it with a command like this:

git filter-branch --index-filter \
              'git rm -r --cached --ignore-unmatch sites/theme/AwesomeTheme'     HEAD

Now you can run git submodule add [email protected] sites/themes/AwesomeTheme

Since the main repository has never seen anything (a.k.a index'd) in sites/themes/AwesomeTheme before, it can now create it.

Absorb

Answer 7

I got it working by doing it the other way around. Starting with an empty repo, adding the submodule in a new folder called "projectfolder/common_code". After that it was possible to add the project code in projectfolder. The details are shown below.

In an empty repo type:

git submodule add url_to_repo projectfolder/common_code

That will create the desired folder structure:

repo
|-- projectfolder
    |-- common_code

It is now possible to add more submodules, and the project code can be added to projectfolder.

I can't yet say why it worked this way around and not the other.

Phemia

Answer 8

Go to the repository folder. Delete relevant submodules from .gitmodules. Select show hidden files. Go to .git folder, delete the submodules from module folder and config.

Affricate

Answer 9

If you are using Ubuntu then remember to delete files from trash also.

In my Case i moved the files to trash.

Earl

Unbelievable, that was my problem indeed. – Celinacelinda

Answer 10

Don't know if this is any useful, though I had the same problem when trying to commit my files from within IntelliJ 15. In the end, I opened SourceTree and in there I could simply commit the file. Problem solved. No need to issue any fancy git commands. Just mentioning it in case anyone has the same issue.

Kor

Answer 11

This happens if the .git file is missing in the target path. It happend to me after I executed git clean -f -d.

I had to delete all target folders showing in the message and then executing git submodule update --remote

Lacto

Careful with the git clean -f -d ! Ensure all files are backuped first !!! – Portugal

I never mentioned that you should execute git clean -f -d – Lacto

Answer 12

In your git dir, suppose you have sync all changes.

rm -rf .git 

rm -rf .gitmodules

Then do:

git init
git submodule add url_to_repo projectfolder

Palpitant

this will delete the whole history of the project ! – Giddings