Making 'git log' ignore changes for certain paths
Asked Answered
H

3

188

How can I make git log only show commits that changed files other than the ones I specify?

With git log, I can filter the commits I see to those that touch a given set of paths. What I want is to invert that filter so that only commits that touch paths other than the specified ones will be listed.

I can get what I want with

git log --format="%n/%n%H" --name-only | ~/filter-log.pl | git log --stdin --no-walk

where filter-log.pl is:

#!/usr/bin/perl
use strict;
use warnings;

$/ = "\n/\n";
<>;

while (<>) {
    my ($commit, @files) = split /\n/, $_;

    if (grep { $_ && $_ !~ m[^(/$|.etckeeper$|lvm/(archive|backup)/)] } @files) {
        print "$commit\n";
    }
}

except I want something somewhat more elegant than that.

Note that I am not asking how to make git ignore the files. These files should be tracked and committed. It's just that, most of the time, I'm not interested in seeing them.

Related question: How to invert git log --grep=<pattern> or How to show Git logs that don't match a pattern. It's the same question except for commit messages rather than paths.

Forum discussion on this subject from 2008: Re: Excluding files from git-diff. This looked promising but the thread seems to have dried up.

Hawsepipe answered 16/4, 2011 at 6:52 Comment(3)
I'm not sure if there is a built-in way, and your perl solution looks pretty decent. If you modify it to accept the paths as command-line arguments, you could just create an alias something like !f() { git log ... | path/to/filter-log.pl "$@" | git log --stdin --no-walk; f, or even wrap that pipeline part up into the script as well.Unequaled
As a workaround, I use find to filter out directories whose commits I do not want to see. If I wanted to ignore log entries from commits made to the root-level directory SiteConfig then I would say: git log `find . -type d -mindepth 1 -maxdepth 1 ! -name *SiteConfig`Repressive
For Git 1.9/2.0 (Q1 2014), see my answer below: git log --oneline --format=%s -- . ":!sub" will work (with the pathspec magic :(exclude) and its short form :!)Complicate
C
348

It is implemented now (git 1.9/2.0, Q1 2014) with the introduction pathspec magic :(exclude) and its short form :! in commit ef79b1f and commit 1649612, by Nguyễn Thái Ngọc Duy (pclouds), documentation can be found here.

You now can log everything except a sub-folder content:

git log -- . ':(exclude)sub'
git log -- . ':!sub'

Or you can exclude specific elements within that sub-folder

  • a specific file:

      git log -- . ':(exclude)sub/sub/file'
      git log -- . ':!sub/sub/file'
    
  • any given file within sub:

      git log -- . ':(exclude)sub/*file'
      git log -- . ':!sub/*file'
      git log -- . ':(exclude,glob)sub/*/file'
    

You can make that exclusion case insensitive!

git log -- . ':(exclude,icase)SUB'

As Kenny Evitt noted

Don't forget to use single quotes or proper escaping in double quotes if you're running git in a bash shell, e.g. ':!sub' or ":\!sub". Otherwise you will run into bash: ... event not found errors


Note: Git 2.13 (Q2 2017) adds a synonym ^ to !

See commit 859b7f1, commit 42ebeb9 (08 Feb 2017) by Linus Torvalds (torvalds).
(Merged by Junio C Hamano -- gitster -- in commit 015fba3, 27 Feb 2017)

pathspec magic: add '^' as alias for '!'

The choice of '!' for a negative pathspec ends up not only not matching what we do for revisions, it's also a horrible character for shell expansion since it needs quoting.

So add '^' as an alternative alias for an excluding pathspec entry.


Note that, before Git 2.28 (Q3 2020), the use of negative pathspec, while collecting paths including untracked ones in the working tree, was broken.

See commit f1f061e (05 Jun 2020) by Elijah Newren (newren).
(Merged by Junio C Hamano -- gitster -- in commit 64efa11, 18 Jun 2020)

dir: fix treatment of negated pathspecs

Reported-by: John Millikin
Signed-off-by: Elijah Newren

do_match_pathspec() started life as match_pathspec_depth_1() and for correctness was only supposed to be called from match_pathspec_depth(). match_pathspec_depth() was later renamed to match_pathspec(), so the invariant we expect today is that do_match_pathspec() has no direct callers outside of match_pathspec().

Unfortunately, this intention was lost with the renames of the two functions, and additional calls to do_match_pathspec() were added in commits

  • 75a6315f74 ("ls-files: add pathspec matching for submodules", 2016-10-07, Git v2.11.0-rc0 -- merge listed in batch #11)
  • 89a1f4aaf7 ("dir: if our pathspec might match files under a dir, recurse into it", 2019-09-17, Git v2.24.0-rc0).

Of course, do_match_pathspec() had an important advantge over match_pathspec() -- match_pathspec() would hardcode flags to one of two values, and these new callers needed to pass some other value for flags.

Also, although calling do_match_pathspec() directly was incorrect, there likely wasn't any difference in the observable end output, because the bug just meant that fill_diretory() would recurse into unneeded directories.

Since subsequent does-this-path-match checks on individual paths under the directory would cause those extra paths to be filtered out, the only difference from using the wrong function was unnecessary computation.

The second of those bad calls to do_match_pathspec() was involved -- via either direct movement or via copying+editing -- into a number of later refactors.

See commits

  • 777b420347 ("dir: synchronize treat_leading_path() and read_directory_recursive()", 2019-12-19, Git v2.25.0-rc0 -- merge)
  • 8d92fb2927 ("dir: replace exponential algorithm with a linear one", 2020-04-01, Git v2.27.0-rc0 -- merge listed in batch #5)
  • 95c11ecc73 ("Fix error-prone fill_directory() API; make it only return matches", 2020-04-01, Git v2.27.0-rc0 -- merge listed in batch #5)

The last of those introduced the usage of do_match_pathspec() on an individual file, and thus resulted in individual paths being returned that shouldn't be.

The problem with calling do_match_pathspec() instead of match_pathspec() is that any negated patterns such as :!unwanted_path will be ignored.

Add a new match_pathspec_with_flags() function to fulfill the needs of specifying special flags while still correctly checking negated patterns, add a big comment above do_match_pathspec() to prevent others from misusing it, and correct current callers of do_match_pathspec() to instead use either match_pathspec() or match_pathspec_with_flags().

One final note is that DO_MATCH_LEADING_PATHSPEC needs special consideration when working with DO_MATCH_EXCLUDE.

The point of DO_MATCH_LEADING_PATHSPEC is that if we have a pathspec like

*/Makefile

and we are checking a directory path like

src/module/component

that we want to consider it a match so that we recurse into the directory because it might have a file named Makefile somewhere below.

However, when we are using an exclusion pattern, i.e. we have a pathspec like

:(exclude)*/Makefile

we do NOT want to say that a directory path like

src/module/component

is a (negative) match.

While there might be a file named Makefile somewhere below that directory, there could also be other files and we cannot pre-emptively rule all the files under that directory out; we need to recurse and then check individual files.

Adjust the DO_MATCH_LEADING_PATHSPEC logic to only get activated for positive pathspecs.


Peter Mortensen adds in the comments:

But it is not sophisticated enough for more specific matching than matching any character(?).

For example, matching numbers in the path (example: "keyboards/keychron/c1_pro", "keyboards/keychron/c2_pro", "keyboards/keychron/c42_pro", "keyboards/keychron/c821_pro", etc.).

That it is far from something resembling regular expressions (if that is the case). Or is there an escape?

True: Git pathspecs do not support regular expressions directly, which means you cannot use them to match specific patterns like numbers or complex strings within paths in a manner similar to regular expressions.

For cases like Peter's examples (keyboards/keychron/c1_pro, keyboards/keychron/c2_pro, keyboards/keychron/c42_pro, keyboards/keychron/c821_pro, etc.), the pathspec magic :(exclude) or :! allows for basic wildcard matching but does not offer the granularity of regular expressions.
You can exclude paths based on simple patterns like wildcards (* for any sequence of characters, ? for any single character) but not with the specificity that regular expressions provide (e.g., matching any path ending with a sequence of numbers followed by _pro).

If you need to filter Git logs based on more complex patterns that require regular expression-like matching, you would typically need to combine git log with other command-line tools that support regular expressions, such as grep or awk.
For example, to list commits affecting paths that match a specific pattern, you might first list all affected paths with git log --name-only, then filter those paths using grep with a regular expression.

However, for Peter's specific use case of excluding paths, you could use a workaround by leveraging the shell's capabilities or scripting. You would list all commits, then filter out commits that only affect the specified patterns. There is no direct escape within Git's pathspec syntax that introduces regular expression capabilities.

Something like:

#!/bin/bash

# Define the pattern to exclude. That uses extended regex for matching any 'keychron' directory followed by 'c', numbers, and '_pro'
exclude_pattern='keyboards/keychron/c[0-9]+_pro'

# Get a list of all commits, showing affected paths, then filter out commits that only modify paths matching the exclude pattern
git log --all --name-only --format="%H" | \
awk '/^[0-9a-f]{40}$/{commit=$0; next} {print commit " " $0}' | \
grep -vE "$exclude_pattern" | \
cut -d' ' -f1 | \
uniq | \
while read commit; do
    git log -1 --format="%h %ad %s" --date=short $commit
done
Complicate answered 12/1, 2014 at 19:47 Comment(19)
Can you do multiple files?Ela
@JustinThomas I believe (not tested yet) that you can repeat that path exclusion pattern multiple time ":(exclude)pathPattern1" ":(exclude)pathPattern2", hence ignoring multiple folders/files.Complicate
Yeah, that worked, I just didn't have 1.9 yet. Super awesome feature for a repo with minified files checked in.Ela
If you're running Git in a Bash shell, use ':!sub' instead to avoid bash: ... event not found errors. ":\!sub" doesn't work.Matrilocal
@KennyEvitt Thank for your edit and comment. I have included the latter in the answer for more visibility.Complicate
Is it possible to configure it s.t. I can permanently exclude, say, the po directory?Sulla
@FrederickNord would you like to keep the po directory content tracked? https://mcmap.net/q/12018/-git-update-index-assume-unchanged-on-directory. If the content is not yet tracked, all you need to do is add po/ to a .gitignore.Complicate
@Complicate I have tested repeating with multiple path exclusion patterns, but it does not quite do what @Hawsepipe asked for, which is to exclude commits which touch .etckeeper or lvm/archive or lvm/backup, even if they also touch other paths. Instead it only excludes commits which only touch those paths. If you have a commit which touches (say) .etckeeper and foobar, it's still displayed. So while this is a great answer which taught me something new, sadly I think it doesn't technically answer the question.Treasonable
@AdamSpiers OK. And I suppose you tested that with the latest Git 2.16.x?Complicate
@Complicate Yes, tested with both 2.16.1.194.gb2e45c695d and an older 2.14.x.Treasonable
For those wondering where the official documentation on this functionality is, see git help glossary (which I found listed in git help -g [which I found suggested in git help]).Visayan
Does it work for --no-index?Bedclothes
@Michael Do you mean git diff --no-index? I do not see a git log --no-index option.Complicate
Yes, sorry. I meant git diff --no-index. I was linked here from a question about that and didn't pay attention to the context. 😅Bedclothes
@Michael OK. You wouldn't have the URL of the question you were linked from, by any chance? That would help for understanding the context of this inquiry.Complicate
Yes. It looks like I was the one who linked it with "Possibly related?" Probably as a reminder to come back and see whether :! syntax worked there.Bedclothes
@Michael OK. I don't think that syntax can work with git diff --no-index, since there is no "tree" to walk. As opposed to git diff, where you can use a negative pathspec. As stated as much in "git diff --no-index exclude files".Complicate
But it is not sophisticated enough for more specific matching than matching any character(?). For example, matching numbers in the path (example: "keyboards/keychron/c1_pro", "keyboards/keychron/c2_pro", "keyboards/keychron/c42_pro", "keyboards/keychron/c821_pro", etc.). Perhaps state that explicitly in the answer? That it is far from something resembling regular expressions (if that is the case). Or is there an escape? That could also be included in the answer (if there is an escape).Rajab
@PeterMortensen Good point. I have updated the answer accordingly.Complicate
D
4

tl;dr: shopt -s extglob && git log !(unwanted/glob|another/unwanted/glob)

If you're using Bash you should be able to use the extended globbing feature to get only the files you need:

$ cd -- "$(mktemp --directory)" 
$ git init
Initialized empty Git repository in /tmp/tmp.cJm8k38G9y/.git/
$ mkdir aye bee
$ echo foo > aye/foo
$ git add aye/foo
$ git commit -m "First commit"
[master (root-commit) 46a028c] First commit
 0 files changed
 create mode 100644 aye/foo
$ echo foo > bee/foo
$ git add bee/foo
$ git commit -m "Second commit"
[master 30b3af2] Second commit
 1 file changed, 1 insertion(+)
 create mode 100644 bee/foo
$ shopt -s extglob
$ git log !(bee)
commit ec660acdb38ee288a9e771a2685fe3389bed01dd
Author: My Name <[email protected]>
Date:   Wed Jun 5 10:58:45 2013 +0200

    First commit

You can combine this with globstar for recursive action.

Declivous answered 5/6, 2013 at 9:1 Comment(1)
This does not show commits affecting files that no longer exist. Very close and nice hack all the same.Hawsepipe
M
-2

You can temporarily ignore the changes in a file with:

git update-index --skip-worktree path/to/file

Going forward, all changes to those files will be ignored by git status, git commit -a, etc. When you're ready to commit those files, just reverse it:

git update-index --no-skip-worktree path/to/file

and commit as normal.

Modiolus answered 24/6, 2011 at 19:4 Comment(1)
This appears to address a slightly different situation. git update-index --skip-worktree does not cause git log to filter commits that have already been made.Hawsepipe

© 2022 - 2024 — McMap. All rights reserved.