Whitelisting and subdirectories in Git

Question

I have created a white-list for text files only.

*
!*.txt

Now, I have an untracked text file in a sub-directory - sub/dir/file.txt, and this is NOT shown (it is ignored). Text files in the root directory are shown, however.

Why is that, and how do I fix it?

Megaton

Related: https://mcmap.net/q/11763/-make-gitignore-ignore-everything-except-a-few-files/78845 – Stirpiculture

Are the files you want to include in a sub directory? If so, this sub directory might be ignored already by the first star in your .gitignore. – Prestonprestress

Yes. These files will be as low in the directory structure as the 11.1.102.55 folder in my example. The nesting is not always the same between folders/products. – Megaton

If it is ok for you, .gitignore can also be inside a sub directory and filter everything inside that sub directory. – Prestonprestress

I know of this functionality. We have hundreds of directories. It would be faster to just explicitly ignore the filetypes we don't want than make one gitignore for each directory. I'm really looking for a way to get whitelist functionality out of the .gitignore file. – Megaton

Basically this is a duplicate of that other question, but this one has the correct answer... – Daric

I don't see how this is a duplicate. The other question asks how to whitelist certain files, while this one asks how to whitelist certain file types including subfolders. Actually, it is more like a duplicate of #8025424. – Powel

@davidkennedy85 I edited the question to highlight the difference, and I've marked it for reopening. – Cissie

fwiw, the granddaddy of duplicate answers on this topic--one that you should look over--would probably be Make .gitignore ignore everything except a few files - very regular activity and several good answers. I'm a particular fan of this answer, which seems clear and thorough. – Derogatory

Answer 1

If you try it that way, it'll fail, because you'll end up blacklisting the directories in your structure.

To solve, you want to blacklist everything that is not a directory, and is not one of the file-types you want to commit, while not blacklisting directories.

The .gitignore file that will do this:

# First, ignore everything
*
# Now, whitelist anything that's a directory
!*/
# And all the file types you're interested in.
!*.one
!*.two
!*.etc

Tested this in a three-level structure white-listing for .txt files in the presence of *.one, *.two and *.three files using a .gitignore located in the root directory of the repository - works for me. You won't have to add .gitignore files to all directories in your structure.

Information I used to figure out the answer came from, amongst other things, this (stackoverflow.com).

Gangster

You also need the entry !.gitignore, when using *, don't you? – Catamnesis

See my answer here: #987642 It may be more useful to use /* to blacklist just the top-level folder, but not all folders recursively, which actually means .gitignore shouldn't look into a whitelisted folder's subfolders. – Muscadel

When I ignored everything using *, as in the second line of the example .gitignore file, and then unignored files and directories with the notation !./directory/ and !./file, and then tried to git add those files and directories, Git only added the files: git status showed the files as being staged, but said nothing about the directories. Also, for some reason, the command git branch did not output anything. But when I changed the first ignore pattern to /*, things started working as expected: both files and directories were added on git add, and git branch output master. – Dorotea

Answer 2

A simpler way of achieving this is:

# Ignore all files...
*.*

# ...except the ones we want
!*.txt

This works because gitignore applies patterns that do not start with / to every level below the .gitignore file:

If there is a separator at the beginning or middle (or both) of the pattern, then the pattern is relative to the directory level of the particular .gitignore file itself. Otherwise the pattern may also match at any level below the .gitignore level.

If you wanted to do this to files inside a directory, things get more complex:

# Ignore all files in all directories inside subdir...
/subdir/**/*.*

# ...except the ones we want
!/subdir/**/*.txt

This works because gitignore has special rules for **:

Two consecutive asterisks ("**") in patterns matched against full pathname may have special meaning:

• A slash followed by two consecutive asterisks then a slash matches zero or more directories. For example, "a/**/b" matches "a/b", "a/x/b", "a/x/y/b" and so on.

The key piece is to make sure you don't ignore directories, because then every file within that directory is ignored regardless of other rules.

Betthel

The comment for *.* is wrong. It should be "Ignore all files/folders which have a period." Glob patterns are not file-aware; they can only be made folder-aware by use of a trailing slash. – Coincidentally

Answer 3

I searched for a long time:

1. Assume I have a large folder structure with ~100.000 directories recursively nested. In those folders, there're about 30.000 files of type .txt (in my case: type *.md). Next to these *.md files, there're, lets say, 500GB of (a million+) files that I don't want to track.

2. I want git to track only .txt (or *.md) files in all folders and subdirs.

The correct answer should be: this is not possible in Git.

What I did instead:

[edit: did also not work - I tried to create a folder with symlinks (or hardlinks) and use git there, but git doesn't follow symlinks and overwrites hardlinks. Doh!]

Borchers

Answer 4

If you want to ignore everything in the repository except .txt files , .gitignore should be like:

# Ignore everything
*
# Whitelist every directories, so we can recurse into them
!*/
# Whitelist .gitignore and .txt
!.gitignore
!*.txt

But if you want to ignore just one folder of your repository and whitelist .txt files in any directory or subdirectory:

# Recursively ignore every files in my_folder
/my_folder/**
# Recursively whitelist every directories in my_folder
!/my_folder/**/
# Whitelist .txt
!*.txt

Superimpose