grep for stuff in multiple git repositories

Question

So I've inherited a fairly large code base from some other developers, with code stored in various git repositories.

Sometimes, it's hard to know which project a particular piece of code might lie in, or if that piece of code even exists in git.

What I want to be able to do is grep ALL the projects for some particular piece of text.

I'm using gitosis, so all the git repositories are stored in /home/git/repositories with a structure like:

/home/git/repositories
  |- project1
    |- HEAD
    |- branches
    |- config
    |- description
    |- hooks
    |- info
    |- objects
    |- refs
  |- project2
    |- ...

I've tried doing a recursive grep for stuff in the objects directories like this:

grep -Hr "text" /home/git/repositories/*/objects

This fails to work as I intend of course, because the objects are stored in git's custom format.

What do?

Sallust

Answer 1

Use git grep with a ref or --no-index:

cd /home/git/repositories
for i in *; do ( cd $i; git grep text HEAD ); done

Pamela

If you're looking for a particular piece of code, as opposed to text in a commit message, you could also use git log -S"<string>" here. It's called a pickaxe search. – Pine

the problem is that the folders in /home/git/repositories aren't actually git working trees so git grep doesn't work on them. – Sallust

@python_noob git grep does work in a bare repo if you specify a ref (eg, HEAD) – Pamela

Well, that's frustrating. I just went searching for a way to do this that is better than a simple shell wrapper, and the only answer I find is my own from 7 years ago! Surely there is some better tooling now! – Pamela

I've filed a feature request: public-inbox.org/git/… – Keel

Answer 2

I know its old question but if you use command line you can add this to bash_profile or bashrc

ggrep() {
    find . -type d -name .git | while read line; do
        (
        cd $line/..
        cwd=$(pwd)
        echo "$(tput setaf 2)$cwd$(tput sgr0)"
        git grep -n "$@"
        )
    done
}

basic gist of above function is to search of all directories that contains .git and output first that directory then file along with line number where that token occurs

then go to /home/git/repositories and search using

ggrep "InvalidToken"

it will output like this

/home/git/org/repo1
/home/git/org/repo2
/home/git/org/repo3
/home/git/org/repo3
lib/v3/Utility.pm:59:         code              => 'InvalidToken',
lib/v3/Utility.pm:142:        code              => "InvalidToken",

you can also pass flags like ggrep -i "search" (for case insensitive search)

Junko

Nice solution. One observation from my usage is that because I know the repositories are all direct descendants of my current folder I added -maxdepth 2 to the find command's arguments, which makes this execute a lot quicker for me. – Bernie

I had to add --no-pager to see output as described in this answer (all at once instead of reviewing results folder by folder) – Mahla

Answer 3

Use multi. It was written specifically to git grep through multiple repositories at once.

$ ls
vim spring-framework gradle phantomjs
$ multi -i "fantastic"
vim
====================================================
runtime/doc/quotes.txt:VIM 4.5 is really a fantastic editor.  It has sooooo many features and more
runtime/doc/quotes.txt:fantastic it is! (Tony Nugent, Australia)
spring-framework
====================================================
gradle
====================================================
subprojects/docs/src/docs/userguide/ant.xml:        simply by relying on Groovy, and the fantastic <literal>AntBuilder</literal>.
subprojects/docs/src/docs/userguide/buildScriptsTutorial.xml:            relying on Groovy. Groovy is shipped with the fantastic <literal>AntBuilder</literal>. Using Ant tasks
subprojects/docs/src/docs/userguide/ideSupport.xml:            if you do this you have a fantastic IDE support for developing Gradle scripts. Of course if you use
phantomjs
====================================================
test/ghostdriver-test/fixtures/common/macbeth.html:<A NAME=1.3.55>Are ye fantastical, or that indeed</A><br>
test/ghostdriver-test/fixtures/common/macbeth.html:<A NAME=1.3.148>My thought, whose murder yet is but fantastical,</A><br>

Amendment

Multi is still a great tool for this use-case in 2021. – Marjoram