List all files that are not ignored by .gitignore

Question

I would like to list all files that are not ignored by .gitignore, ie all source files of my repository.

ag does it well by default, but I'm not aware of an approach that works without installing additional software.

git ls-files without options works almost well but doesn't take into account the files that have been modified/created, for example if I create a new file bar without commiting it, git ls-files doesn't show that file.

Nancynandor

did you mean ag -l? – Fulfill

Answer 1

git status --short| grep  '^?' | cut -d\  -f2- 

will give you untracked files.

If you union it with git ls-files, you've got all unignored files:

( git status --short| grep '^?' | cut -d\  -f2- && git ls-files ) | sort -u

you can then filter by

( xargs -d '\n' -- stat -c%n 2>/dev/null  ||: )

to get only the files that are stat-able (== on disk).

Regin

What is the ||: part ?? – Nancynandor

: == true; to ignore the nonzero exit status of stat, which is to be expected unless all the files on stdin really do exist on disk – Regin

(2>/dev/null and ||: potentially hides other failures in stat such as EPERM, ENOMEM, or EIO fails, but you shouldn't get any of those unless your repo and your system are in a very weird state, in which case you'd be getting lots of errors from other utilities) – Regin

p.s. It seems that git status --short will give directory name if a new directory and some files under it is added, instead of directory + file name. – Trela

how would you chain the xargs part with the rest? or store the results in an array? – Potentiate

@Potentiate Using |, i.e., ( git status --short| grep '^?' | cut -d\ -f2- && git ls-files ) | sort -u | ( xargs -d '\n' -- stat -c%n 2>/dev/null ||: ) – Regin

@PSkocik oh ok, I see. The parenthesis around the xargs -d ... command threw me off there. This is working very well, thanks! I also managed to store the results in an array by using command substitution: project_files=$(( git status --short | grep '^?' | cut -d\ -f2- && git ls-files ) | sort -u | ( xargs -d '\n' -- stat -c%n 2>/dev/null ||: )) – Potentiate

From the documentation, it seems git ls-files -c -m -o should do the trick. Or am I mistaken ? – Hyperbaton

Answer 2

git ls-files --cached --others --exclude-standard will give you the info you want.

• --cached -- Show files that git knows about. (Plain git ls-files with no options is equivalent to passing this option.)
• --others -- Show files that git doesn't know about. This includes ignored files by default, but...
• --exclude-standard -- Don't include ignored files in the output.

Hernardo

Answer 3

Here is another way, using git check-ignore which you may find it cleaner:

find . -type f

will give you all the files in the current folder.

find . -type f -not -path './.git/*'

will give you all the files, with the exception of everything in .git folder, which you definitely don't want to include. then,

for f in $(find . -type f -not -path './.git/*'); do echo $f; done

is the same as above. Just a preparation to the next step, which introduce the condition. The idea is to use the git check-ignore -q which returns exit 0 when a path is ignored. Hence, here is the full command

for f in $(find . -type f -a -not -path './.git/*'); do
    if ! $(git check-ignore -q $f); then echo $f; fi
done

Eyot

Nice! Just note that this is very slow if you have large directories that are ignored, e.g. node_modules. You have to keep adding the large directories, which somewhat defeats the purpose of trying to automatically respect the .gitignore file. But just adding -not -path './node_modules/*' is good enough for my use. – Halcyon

This one-liner is much faster since it avoids repeat calls to git check-ignore. It also uses -prune which should be more efficient: find . -name '.git' -prune -or -type f -print | git check-ignore --verbose --non-matching --stdin | sed -n 's/^::\t//p' – Holtorf

Answer 4

You can use fd.

The command to run is:

fd -H

Please note that in case there is no .git directory and only a .gitignore file (ex. bare repo), you have to specifically tell where the .gitignore file is.

fd -H --ignore-file .gitignore

For more details please check here.

Fulfill

very good, I have been seeking this solution about 3 days! I used rsync dry-run before, but it just ignores some .gitignore expressions – Lubet

Answer 5

Shorter variant of Uri's answer:

• to find ignored files:

find . -not -path ./.git/\* -type f -exec git check-ignore -q {} \; -print

• to find non-ignored files:

find . -not -path ./.git/\* -type f -not -exec git check-ignore -q {} \; -print

Feuillant

Answer 6

-t option shows "tags" that indicate file status, and "R" indicates the file is already deleted. If we exclude files with both "R" and non-"R" line, we get what we want.

git ls-files -dcmot --exclude-standard | sed "s/^[HSMCK?U]/X/" | uniq | sed "s/^R/X/" | uniq -u | sed "s/^X //"

Stoush