How to track untracked content?

Question

See below the solid line for my original question.

I have a folder in my local directory that is untracked. When I run git status, I get:

Changed but not updated:
modified:   vendor/plugins/open_flash_chart_2 (modified content, untracked content)

When I type git add vendor/plugins/open_flash_chart_2 then try git status again, it still says untracked. What's going on?

---

Here is a simple summary of my latest half hour:

• Discovered that my Github repo is not tracking my vendor/plugins/open_flash_chart_2 plugin. Specifically, there's no content and it's showing a green arrow on the folder icon.

• Tried git submodule init

  No submodule mapping found in .gitmodules for path 'vendor/plugins/open_flash_chart_2'
  

• Tried git submodule add git://github.com/korin/open_flash_chart_2_plugin.git vendor/plugins/open_flash_chart_2

  vendor/plugins/open_flash_chart_2 already exists in the index
  

• git status

  modified: vendor/plugins/open_flash_chart_2 (untracked content)
  

• Hunted for any file named .gitmodules in my repository/local directory but couldn't find one.

What do I have to do to get my submodules working so git can start tracking properly?

---

This may be unrelated (I include it in case it helps), but every time I type git commit -a rather than my usual git commit -m "my comments", it throws up an error:

E325: ATTENTION
Found a swap file by the name ".git\.COMMIT-EDITMSG.swp"
         dated: Thu Nov 11 19:45:05 2010
     file name: c:/san/project/.git/COMMIT_EDITMSG
      modified: YES
     user name: San   host name: San-PC
    process ID: 4268
While opening file ".git\COMMIT_EDITMSG"
         dated: Thu Nov 11 20:56:09 2010
  NEWER than swap file!  
Swap file ".git\.COMMIT_EDITMSG.swp" already exists!
[O]pen Read-Only, (E)dit anyway, (R)ecover, (D)elete it, (Q)uit, (A)bort:
Swap file ".git\.COMMIT_EDITMSG.swp" already exists!
[O]pen Read-Only, (E)dit anyway, (R)ecover, (D)elete it, (Q)uit, (A)bort:

I am a complete newbie at Github and despite trying to go through the documentation, I'm a bit stumped by these particular problems. Thank you.

Docker

I'm confused about a few things. One: why do you keep saying github? everything you're talking about sounds local (except that green arrow - I know nothing about that). Is that right, or is this a question about pushing updates to github? Two: have you actually done anything to create this plugin as a submodule? A submodule is a git repository in its own right. You create it separately, then add it into the superproject. – Permian

Hm, the output of git status suggests that there is indeed a submodule there... but you really don't have a .gitmodules file? Honestly, if this plugin has its own repo, and you know what version of it you want, you could just remove that directory from your project, make sure there's no entry in .gitmodules, and go through the submodule setup: git submodule add, git submodule update --init. – Permian

Jefromi - I haven't actually mentioned Github more than twice - the first time is because the green arrow appears on top of the yellow folder image in the actual Github repository for open_flash_chart_2. I'll happily edit the answer to make it clearer. – Docker

It seems that git is ...kindof... treating open_flash_chart_2 as a submodule even though I don't even particularly want it to (and I have done nothing to create any submodule). How can I just get this folder to track properly without setting up a separate entity? (git add . isn't doing it) – Docker

Wait, so do you want it as a submodule or not? It sounds like either way you might need to clean out your index - git reset HEAD will restore the whole thing to the state from the current commit. – Permian

Is this a public github repo? – Permian

Hi Jefromi, it's a private repo. I just want vendors/plugins/open_flash_chart_2 to be uploaded and tracked in my repo (I don't care at all that it's a submodule). – Docker

@sscirrus: You should definitely have a preference, submodule or not. If this is an external thing you aren't going to work on, but which you may want updates from, it should be a submodule. If you don't care about getting updates from its original repo, and might want to hack on the source yourself, it should not be a submodule. – Permian

Maybe you've got something left behind in a gitconfig. I found this: support.github.com/discussions/repos/… – Permian

Jefromi, in that case I would prefer not to have a submodule - I'd rather just have the code stably sit in my vendors/plugins folder. How can I just get this folder tracking like all the others? (I'm reading progit.org right now) – Docker

Clarification: git status shows modified: vendor/plugins/open_flash_chart_2 (modified content, untracked content). When I type git add vendor/plugins/open_flash_chart_2, nothing seems to happen (it stays untracked). – Docker

This question seems to be purely about git submodules. I removed some tags because this question isn't directly related to rails or github, it would apply to all git use cases. And the tracking tag seems to be used for the 'website visitor tracking' topics, so that would not apply either. – Diphenylhydantoin

It might be easier to to all this back-and-forth in a chat room. chat.stackoverflow.com/rooms/55/git it fairly quiet, or we could make a temporary room for this question. – Burk

Answer 1

You have added vendor/plugins/open_flash_chart_2 as “gitlink” entry, but never defined it as a submodule. Effectively you are using the internal feature that git submodule uses (gitlink entries) but you are not using the submodule feature itself.

You probably did something like this:

git clone git://github.com/korin/open_flash_chart_2_plugin.git vendor/plugins/open_flash_chart_2
git add vendor/plugins/open_flash_chart_2

This last command is the problem. The directory vendor/plugins/open_flash_chart_2 starts out as an independent Git repository. Usually such sub-repositories are ignored, but if you tell git add to explicitly add it, then it will create an gitlink entry that points to the sub-repository’s HEAD commit instead of adding the contents of the directory. It might be nice if git add would refuse to create such “semi-submodules”.

Normal directories are represented as tree objects in Git; tree objects give names, and permissions to the objects they contain (usually other tree and blob objects—directories and files, respectively). Submodules are represented as “gitlink” entries; gitlink entries only contain the object name (hash) of the HEAD commit of the submodule. The “source repository” for a gitlink’s commit is specified in the .gitmodules file (and the .git/config file once the submodule has been initialized).

What you have is an entry that points to a particular commit, without recording the source repository for that commit. You can fix this by either making your gitlink into a proper submodule, or by removing the gitlink and replacing it with “normal” content (plain files and directories).

Turn It into a Proper Submodule

The only bit you are missing to properly define vendor/plugins/open_flash_chart_2 as a submodule is a .gitmodules file. Normally (if you had not already added it as bare gitlink entry), you would just use git submodule add:

git submodule add git://github.com/korin/open_flash_chart_2_plugin.git vendor/plugins/open_flash_chart_2

As you found, this will not work if the path already exists in the index. The solution is to temporarily remove the gitlink entry from the index and then add the submodule:

git rm --cached vendor/plugins/open_flash_chart_2
git submodule add git://github.com/korin/open_flash_chart_2_plugin.git vendor/plugins/open_flash_chart_2

This will use your existing sub-repository (i.e. it will not re-clone the source repository) and stage a .gitmodules file that looks like this:

[submodule "vendor/plugins/open_flash_chart_2"]
    path = vendor/plugins/open_flash_chart_2
    url = git://github.com/korin/open_flash_chart_2_plugin.git vendor/plugins/open_flash_chart_2

It will also make a similar entry in your main repository’s .git/config (without the path setting).

Commit that and you will have a proper submodule. When you clone the repository (or push to GitHub and clone from there), you should be able to re-initialize the submodule via git submodule update --init.

Replace It with Plain Content

The next step assumes that your sub-repository in vendor/plugins/open_flash_chart_2 does not have any local history that you want to preserve (i.e. all you care about is the current working tree of the sub-repository, not the history).

If you have local history in the sub-repository that you care about, then you should backup the sub-repository’s .git directory before deleting it in the second command below. (Also consider the git subtree example below that preserves the history of the sub-repository’s HEAD).

git rm --cached vendor/plugins/open_flash_chart_2
rm -rf vendor/plugins/open_flash_chart_2/.git # BACK THIS UP FIRST unless you are sure you have no local changes in it
git add vendor/plugins/open_flash_chart_2

This time when adding the directory, it is not a sub-repository, so the files will be added normally. Unfortunately, since we deleted the .git directory there is no super-easy way to keep things up-to-date with the source repository.

You might consider using a subtree merge instead. Doing so will let you easily pull in changes from the source repository while keeping the files “flat” in your repository (no submodules). The third-party git subtree command is a nice wrapper around the subtree merge functionality.

git rm --cached vendor/plugins/open_flash_chart_2
git commit -m'converting to subtree; please stand by'
mv vendor/plugins/open_flash_chart_2 ../ofc2.local
git subtree add --prefix=vendor/plugins/open_flash_chart_2 ../ofc2.local HEAD
#rm -rf ../ofc2.local # if HEAD was the only tip with local history

Later:

git remote add ofc2 git://github.com/korin/open_flash_chart_2_plugin.git
git subtree pull --prefix=vendor/plugins/open_flash_chart_2 ofc2 master

git subtree push --prefix=vendor/plugins/open_flash_chart_2 [email protected]:me/my_ofc2_fork.git changes_for_pull_request

git subtree also has a --squash option that lets you avoid incorporating the source repository’s history into your history but still lets you pull in upstream changes.

Burk

Chris, I just tried rm -rf vendor/plugins/open_flash_chart_2/.git and it says 'rm' is not recognized. Then I tried git rm -rf vendor/plugins/open_flash_chart_2/.git and it said fatal: pathspec 'vendor/plugins/open_flash_chart_2/.git' did not match any files (but I can get there in windows explorer!). – Docker

We discussed the problem a bit in chat.The OP was able to remove the sub-repository’s .git dir and re-add the files “flatly” (the first “Plain Content” option). – Burk

This saved my ass so much the other day. I had accidentally added something with a .git directory in it to my git repo, and then git kinda-sorta thought it had a submodule, which was causing my git svn dcommits to freak out. I would upvote you 3 times if I could. – Bestead

Want to point, the third-party plugin subtree is not to be confused with subtree merge. It's not a wrapper. Author says so :P – Crash

@NebulaFox: While git subtree is not just a wrapper around subtree merges, it certainly does use the same techniques as the standard “subtree merge” procedure (i.e. git readtree --prefix=path and subtree merges: git merge with -s subtree or -Xsubtree=path). It has nice extras “on top”, too: --squash mode, the split command, and the push and pull helpers. – Burk

@Docker git rm will not work as that .git directory is not staged to git... You have to run rm -rf ..../.git, but that doesn't work for you, because as you said you have Windows. So just open the directory and delete that .git folder within Explorer :) – Improvvisatore

I had a similar issue but I had many repos that needed to be added as submodules. I ended up writing a script to automate the process. It can be found here: gist.github.com/landaire/25d231fca4b3af62d7cd – Trisect

Answer 2

I just had the same problem. The reason was because there was a subfolder that contained a ".git" folder. Removing it made git happy.

Openhearted

Yes, this happened to me in AndroidStudio. The git repo had already been created in the /app folder while I tried to create a git repo one folder up. Thanks. – Troyes

I had an angular cli created app under many other folders and that one folder had a .git folder :/ That is the real problem not submodule stuff! – Neutralism

this works for me ... I have try the git add many times ,but nothing happened. then I remove the .git folder – Antibaryon

But what if I still want to track my subfolder ? What should I do ? – Smtih

Have the same case. Still wanna track that subfolder that sadly contained also a .git folder. After deleting this sufolder .git folder it is not tracked anymore in my main .git folder ... .:( – Enscroll

Simple and efficient. Thanks – Mnemonics

Answer 3

1. I removed the .git directories from those new directories (this can create submodule drama. Google it if interested.)
2. I then ran git rm -rf --cached /the/new/directories
3. Then I re-added the directories with a git add . from above

Reference URL https://danielmiessler.com/blog/git-modified-untracked/#gs.W0C7X6U

Sarisarid

what is '/the/new/directories'? How can you just copy example of another page without giving a explanation? – Eisegesis

Answer 4

To point out what I had to dig out of Chris Johansen's chat with OP (linked from a reply to an answer):

git add vendor/plugins/open_flash_chart_2 # will add gitlink, content will stay untracked

git add vendor/plugins/open_flash_chart_2/ # NOTICE THE SLASH!!!!

The second form will add it without gitlink, and the contents are trackable. The .git dir is conveniently & automatically ignored. Thank you Chris!

Arleta

Answer 5

I use the trick suggested by Peter Lada all the time, dubbed as "fake submodules":

http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb

It's very useful in several scenarios (p.e. I use it to keep all my Emacs config in a repository, including the current HEAD of all git repositories inside the elpa/el-get package directories, so I could easily roll back/forward to a known working version when some update breaks something).

Sevenup

Answer 6

I solved this issue by deleting .git file from my subfolder.

1. First delete .git file from your subfolder
2. Then remove your subfolder from git by running this code, git rm -rf --cached your_subfolder_name
3. Then again add your folder by git add . command

Krauss

Answer 7

https://git-scm.com/book/en/v2/Git-Tools-Submodules

I think you should read this to learn a little about submodule. It's well-written, and it doesn't take much time to read it.

Evanescent

Answer 8

I had the same problem with a big project with many submodules. Based on the answers of Chris Johnsen here and VonC here I build a short bash script which iterates through all existing gitlink entries and adds them as proper submodules.

#!/bin/bash

# Read all submodules in current git
MODULES=`git ls-files --stage | grep 160000`

# Iterate through every submodule path
while read -r MOD; do
  # extract submodule path (split line at whitespace and take string with index 3)
  ARRIN=(${MOD})
  MODPATH=${ARRIN[3]}

  # grep module url from .git file in submodule path
  MODURL=`grep "url = " $MODPATH/.git/config`
  MODURL=${MODURL##*=}

  # echo path and url for information
  echo $MODPATH
  echo $MODURL

  # remove existing entry in submodule index
  git rm --cached $MODPATH
  # add new entry in submodule index
  git submodule add $MODURL $MODPATH
done <<< "$MODULES"

This fixed it for me, I hope it is of any help.

Triquetrous

Answer 9

This worked out just fine for me:

git update-index --skip-worktree

If it doesn't work with the pathname, try the filename. Let me know if this worked for you too.

Bye!

Wallin

Answer 10

Had the same problem, but it was not solved in this discussion.

I hit also the submodule problem as described in the thread opening.

% git status          
# On branch master
# Changes not staged for commit:
#   modified:   bundle/taglist (untracked content)

Looking at the diff I recognized a -dirty appended to a hash: Reading the docs again, solved the problem for me. http://web.mit.edu/jhawk/mnt/spo/git/git-doc/user-manual.html Look at section "Pitfalls with submodules"

Reason was, there were changes or untracked content within the submodule. I first had to got to the submodule directory, do a "git add" + "git commit" to get all content tracked within the submodule.

Then "git status" on the master stated
% git commit
# On branch master
# Changes not staged for commit:
#   modified:   bundle/taglist (new commits)

Now this new HEAD from the submodule could be commited to the master module.

Daye

Answer 11

I recently encountered this problem while working on a contract project(deemed classified). The system in which I had to run the code did not have internet access, for security purposes of course, and so installing dependencies, using composer and npm, was becoming huge pain.

After much deliberation with my colleague, we decided to just wing it and copy paste our dependencies rather than doing composer install or npm install.

This led us to NOT add vendors and npm_modules in gitignore. This is when I encountered this problem.

Changed but not updated:
modified:   vendor/plugins/open_flash_chart_2 (modified content, untracked content)

I googled this a bit and found this helpful thread on SO. Not being too much of a pro in Git, and being a little intoxicated while working on it, I just searched for all the submodules in the vendors folder

find . -name ".git"

This gave me some 4-5 dependencies that had git on them. I removed all these .git folders and voila, it worked. I knows it's hack, and not a very geeky one anyways. O Gods of SO, please forgive me! Next time I promise to read up on gitlinks and obey O mighty Linus Tovalds.

Erastianism

Answer 12

This question has been answered already, but thought I'd add to the mix what I found out when I got these messages.

I have a repo called playground that contains a number of sandbox apps. I added two new apps from a tutorial to the playground directory by cloning the tutorial's repo. The result was that the new apps' git stuff pointed to the tutorial's repo and not to my repo. The solution was to delete the .git directory from each of those apps' directories, mv the apps' directories outside the playground directory, and then mv them back and run git add .. After that it worked.

Outwork

Answer 13

First go to the Directory : vendor/plugins/open_flash_chart_2 and DELETE

---

THEN :

git rm --cached vendor/plugins/open_flash_chart_2  
git add .  
git commit -m "Message"  
git push -u origin master  

git status  

OUTPUT

On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean

Piggy

Answer 14

In my case the problematic module should have been added without leading slash / in the path.

Wrong:

[submodule "/sources/meta-security"]
        path = /sources/meta-security
        url = [email protected]:yocto/meta-security.git

Correct:

[submodule "sources/meta-security"]
        path = sources/meta-security
        url = [email protected]:yocto/meta-security.git

After the change the following command resolved the issue:

git submodule update --init

Opuntia

Answer 15

Only thing that worked for me

• Deleting .git folder inside that sub-directory
• then moving that sub-directory out from main-directory
• git add .
• git commit -m "cleaning up"
• git push
• moving that sub-directory back into main-directory
• again git add .
• git commit -m "message"
• git push

Now it is working

Mechanotherapy