How to git grep the main repository and all its submodules?

Question

I found this at a blog to grep project and submodules:

[alias]
  sgrep = "!f() { git grep \"$1\"; git submodule foreach \"git grep '$1'; true\" | grep -B 1 \"$1\"; }; f"

I updated my ~/.gitconfig to include it, but when I use the alias, it says:

fatal: bad config file line 9 in /home/myname/.gitconfig

What am I doing wrong?

Hypochlorite

Answer 1

Generally, similar to "GitConfig: bad config for shell command", it may be because the '\' character is interpreted by git first, before being executed in a shell.

But in your case, this should work as expected because of the surrounding double-quotes.
It does work on my git 1.8.4 on Ubuntu.

So simplify your .gitconfig and check if works better if it contains your alias (and nothing) else.
If that does work, then the issue is elsewhere in your config file.

Marcelenemarcelia

Answer 2

With Git 2.12 (Q1 2017), you won't need any git submodule foreach trick.
Simply:

 git grep -e "bar" --recurse-submodules

See commit e6fac7f, commit 74ed437, commit 0281e48, commit 4538eef, commit 9ebf689, commit f9f4256, commit 5688c28 (16 Dec 2016), and commit 4ac9006, commit 7241764, commit a1ae484, commit 05b458c (12 Dec 2016) by Brandon Williams (mbrandonw).
See commit e9a379c (21 Dec 2016) by Johannes Sixt (j6t).
^{(Merged by Junio C Hamano -- gitster -- in commit 55d128a, 18 Jan 2017)}

grep: optionally recurse into submodules

Allow grep to recognize submodules and recursively search for patterns in each submodule.
This is done by forking off a process to recursively call grep on each submodule.

Recursion only occurs for submodules which have been initialized and checked out by the parent project. If a submodule hasn't been initialized and checked out it is simply skipped.

In order to support the existing multi-threading infrastructure in grep, output from each child process is captured in a strbuf so that it can be later printed to the console in an ordered fashion.

To limit the number of theads that are created, each child process has half the number of threads as its parents (minimum of 1), otherwise we potentially have a fork-bomb.

The git grep man page now includes:

--recurse-submodules

Recursively search in each submodule that has been initialized and checked out in the repository.
When used in combination with the <tree> option the prefix of all submodule output will be the name of the parent project's <tree> object.

---

With Git 2.14.x/2.15 (Q3 2017), "git grep --recurse-submodules" has been reworked to give a more consistent output across submodule boundary (and do its thing without having to fork a separate process).

See commit f9ee2fc, commit 2184d4b, commit 34e2ba0, commit 91b8348, commit 8fa2915, commit f20e7c1, commit b22e51c, commit 4c0eeaf (02 Aug 2017), and commit ba43964, commit 3f13877 (18 Jul 2017) by Brandon Williams (mbrandonw).
^{(Merged by Junio C Hamano -- gitster -- in commit 5aa0b6c, 22 Aug 2017)}

That means the internal --parent-basename <basename> option of git grep does no longer exists.

---

Make sure to use Git 2.21 (Q1 2019), as a bug has been fixed regarding cywin paths: see "Cygwin uses relative and absolute path for environment variables".

---

Actually, before Git 2.23.1/2.24 (Q4 2019), "git grep --recurse-submodules" that looks at the working tree files looked at the contents in the index in submodules, instead of files in the working tree.

That has been fixed.

See commit 6a289d4 (30 Jul 2019) by Matheus Tavares (matheustavares).
^{(Merged by Junio C Hamano -- gitster -- in commit 3071797, 22 Aug 2019)}

grep: fix worktree case in submodules

Running git-grep with --recurse-submodules results in a cached grep for the submodules even when --cached is not used.
This makes all modifications in submodules' tracked files be always ignored when grepping.

Solve that making git grep respect the cached option when invoking grep_cache() inside grep_submodule().
Also, add tests to ensure that the desired behavior is performed.

---

"git grep --no-index" should not get affected by the contents of the .gitmodules file but when "--recurse-submodules" is given or the "submodule.recurse" variable is set, it did.

No more, with Git 2.25.1 (Feb. 2020).
Now these settings are ignored in the "--no-index" mode.

See commit c56c48d (30 Jan 2020) by Philippe Blain (phil-blain).
^{(Merged by Junio C Hamano -- gitster -- in commit 556ccd4, 12 Feb 2020)}
See discussion.

grep: ignore --recurse-submodules if --no-index is given

^{Helped-by: Junio C Hamano
Signed-off-by: Philippe Blain}

Since grep learned to recurse into submodules in 0281e487fd ("grep: optionally recurse into submodules", 2016-12-16, Git v2.12.0-rc0 -- merge listed in batch #6), using --recurse-submodules along with --no-index makes Git die().

This is unfortunate because if submodule.recurse is set in a user's ~/.gitconfig, invoking git grep --no-index either inside or outside a Git repository results in:

fatal: option not supported with --recurse-submodules

Let's allow using these options together, so that setting submodule.recurse globally does not prevent using git grep --no-index.

Using --recurse-submodules should not have any effect if --no-index is used inside a repository, as Git will recurse into the checked out submodule directories just like into regular directories.

Marcelenemarcelia

See also https://mcmap.net/q/743736/-cygwin-uses-relative-and-absolute-path-for-environment-variables, since a regression for cygwin users was introduced with commit 05b458c, "real_path: resolve symlinks by hand", Dec. 2016, Git v2.12.0-rc0. – Marcelenemarcelia