Searching subversion history (full text)

Question

Is there a way to perform a full text search of a subversion repository, including all the history?

For example, I've written a feature that I used somewhere, but then it wasn't needed, so I svn rm'd the files, but now I need to find it again to use it for something else. The svn log probably says something like "removed unused stuff", and there's loads of checkins like that.

Edit 2016-04-15: Please note that what is asked here by the term "full text search", is to search the actual diffs of the commit history, and not filenames and/or commit messages. I'm pointing this out because the author's phrasing above does not reflect that very well - since in his example he might as well be only looking for a filename and/or commit message. Hence a lot of the svn log answers and comments.

Illicit

Apache Subversion 1.8 accepts --search argument for svn log command. See my answer at https://mcmap.net/q/158584/-searching-subversion-history-full-text – Generic

svn log --search does not perform a full text search as required by @rjmunro, but only searches author, date, log message and list of changed paths. – Shears

Answer 1

git svn clone <svn url>

git log -G<some regex>

Bugger

Good point. The gitk GUI does this kind of search very well, and it's easy to use the git svn tools to access an svn repository. Admitedly, I've moved entirely to using git since the time I asked this question, so accepting this answer might be a bit specific to me. – Illicit

If you want to just search the commit messages, use git log --grep regex. – Zavras

Be advised, this can take a while depending on the size of your repository. For me it took more than an hour. – Presume

More git grep examples in this answer – Garganey

Does this really search through the whole history? – Hanse

@rjmunro: A simple git grep < some regex > as given in the answer does not search the history at all. And thus this doesn't answer your question at all. – Shears

@Shears Converting to git and searching with the gitk gui worked for me. I assumed this answer was the equivalent command line way to do it. Actually it looks like the equivalent command is git log -S (from this answer), but git log -G might be even better. I'm editing this answer. – Illicit

I downvoted this solution since converting a big SVN repository to GIT is often not feasible or would take much too long. It's like recommending Java when having a question about a C# language construct. – Pathos

You may need to install an additional package for this command. On Ubuntu, you want to apt-get install git-svn. – Brennan

Doesn't "svn log --verbose --diff | grep ..." buy your roughly the same functionality without having to use git? – Interknit

@Interknit It roughly is the same functionality. It is very good in so far you work with svn alone. I find this idea to utilize git to search for commits ridiculous. The trouble starts when you want to see more than just the line containing the term you searched for. grep can print a number of lines before the finding and after but you never know how many lines you need (to find the revision number at the top or the whole comment to the bottom). The output is hard to read. – Presidium

This is searching with git, not searching with SVN as per the question. This is the equivalent of providing a Python script to answer a C question - it works, but it doesn't answer the original question. – Protection

Answer 2

Update April, 2022

VisualSVN Server 5.0 comes with a new full-text search feature that allows you to search through the contents and history of your repositories in the web interface. Try out the feature on the demo server.

---

Old answer

svn log in Apache Subversion 1.8 supports a new --search option. So you can search Subversion repository history log messages without using 3'rd party tools and scripts.

svn log --search searches in author, date, log message text and list of changed paths.

See SVNBook | svn log command-line reference.

Generic

Handy, but not full-text search. I'm sticking with the git-svn answer :-) – Illicit

Not that at the moment svn repos on googlecode are still running on svn 1.6... see: code.google.com/p/support/wiki/…? But, updating your client to 1,8 (and svn upgrade of any checked out repo) will allow you to use svn log --search on the repo... – Tsingyuan

The working copy needs all the updates but this command lists the whole change including revision number, changed files and comment. How is it not full-text? – Presidium

Answer 3

If you are running Windows have a look at SvnQuery. It maintains a full text index of local or remote repositories. Every document ever committed to a repository gets indexed. You can do google-like queries from a simple web interface.

Kier

It would be great if SvnQuery was still maintained but sadly it died and now it just doesn't work at all. – Vinery

I found a working clone (?) at github.com/kalyptorisk/svnquery/releases – Winchell

Answer 4

I'm using a small shellscript, but this only works for a single file. You can ofcourse combine this with find to include more files.

#!/bin/bash
for REV in `svn log $1 | grep ^r[0-9] | awk '{print $1}'`; do 
  svn cat $1 -r $REV | grep -q $2
  if [ $? -eq 0 ]; then 
    echo "$REV"
  fi 
done

If you really want to search everything, use the svnadmin dump command and grep through that.

Nipissing

I had to remove the "r" from revisions numbers with: awk '{print substr($1,2,length($1))}' and remove the grep "-q" option, to actually show the matches. – Reeducate

strings myDump.txt | grep "turtle fwd 10" – Sir

And this is why we embrace git. – Zavras

might want to do last grep -i for ignore case and drop the -q to actually see the row that matched – Whatsoever

Saving the repository content with svnadmin dump and searching, worked for me. – Saucedo

Answer 5

The best way that I've found to do this is with less:

svn log --verbose | less

Once less comes up with output, you can hit / to search, like VIM.

Edit:

According to the author, he wants to search more than just the messages and the file names. In which case you will be required to ghetto-hack it together with something like:

svn diff -r0:HEAD | less

You can also substitute grep or something else to do the searching for you. If you want to use this on a sub-directory of the repository, you will need to use svn log to discern the first revision in which that directory existed, and use that revision instead of 0.

Kinshasa

That's not full text searching, it's searching the logs and filenames. – Illicit

If that is the case, then you need to use more expressive commit logs. If you want to grep the difference between revisions, that is whole other ball of wax. And I personally do not know a way to do that. – Kinshasa

> svn diff -r0:HEAD > log > less log is my choice on windows. Thanks – Brickey

Answer 6

svn log -v [repository] > somefile.log

for diff you can use the --diff option

svn log -v --diff [repository] > somefile.log

then use vim or nano or whatever you like using, and do a search for what you're looking for. You'll find it pretty quickly.

It's not a fancy script or anything automated. But it works.

Tonjatonjes

AFAICS, That will search commit messages, not the actual diffs. – Illicit

Then use svn log -v --diff [repository] > somefile.log – Tonjatonjes

or just pipe through grep as in zednight's answer – Peccavi

Answer 7

I have been looking for something similar. The best I have come up with is OpenGrok. I have not tried to implement it yet, but sounds promising.

Soak

I've been using OpenGrok for several months, it rocks. – Liminal

Answer 8

While not free, you might take a look at Fisheye from Atlassian, the same folks that bring you JIRA. It does full text search against SVN with many other useful features.

http://www.atlassian.com/software/fisheye/

Whallon

Fisheye's pretty good. As you say, not free, but the <=10 committer license is only $10/year. – Pictor

Currently, 5 users is $10, BUT at only 10 users it jumps to $1,000 ! – Infeld

Answer 9

I just ran into this problem and

svnadmin dump <repo location> |grep -i <search term>

did the job for me. Returned the revision of the first occurrence and quoted the line I was looking for.

Castilian

Works only locally and will take a lot of time if a repository is large. – Generic

Answer 10

I was looking for the same thing and found this:

http://svn-search.sourceforge.net/

Inequity

Answer 11

Use unix utility like grep:

svn log -l <commit limit> --diff | grep -C <5 or more lines> <search message>

or you can save the result of the svn log somewhere, then search through it

Loehr

add --diff to that to get text search of the changes – Peccavi

Answer 12

I don't have any experience with it, but SupoSE (open source, written in Java) is a tool designed to do exactly this.

Wear

Answer 13

I usually do what Jack M says (use svn log --verbose) but I pipe to grep instead of less.

Outworn

That's not full text searching, it's searching the logs and filenames. – Illicit

That is what I usually end up doing, but I've found that with less you can actually see the revision, date, etc instead of just the line in the comment. That is usually what I'm looking for anyway. – Kinshasa

Answer 14

I wrote this as a cygwin bash script to solve this problem.

However it requires that the search term is currently within the filesystem file. For all the files that match the filesystem grep, an grep of all the svn diffs for that file are then performed. Not perfect, but should be good enough for most usage. Hope this helps.

/usr/local/bin/svngrep

#!/bin/bash
# Usage: svngrep $regex @grep_args

regex="$@"
pattern=`echo $regex | perl -p -e 's/--?\S+//g; s/^\\s+//;'` # strip --args
if [[ ! $regex ]]; then
    echo "Usage: svngrep \$regex @grep_args"
else 
    for file in `grep -irl --no-messages --exclude=\*.tmp --exclude=\.svn $regex ./`;     do 
        revs="`svnrevisions $file`";
        for rev in $revs; do
            diff=`svn diff $file -r$[rev-1]:$rev \
                 --diff-cmd /usr/bin/diff -x "-Ew -U5 --strip-trailing-cr" 2> /dev/null`
            context=`echo "$diff" \
                 | grep -i --color=none   -U5 "^\(+\|-\).*$pattern" \
                 | grep -i --color=always -U5             $pattern  \
                 | grep -v '^+++\|^---\|^===\|^Index: ' \
                 `
            if [[ $context ]]; then
                info=`echo "$diff" | grep '^+++\|^---'`
                log=`svn log $file -r$rev`
                #author=`svn info -r$rev | awk '/Last Changed Author:/ { print $4 }'`; 

                echo "========================================================================"
                echo "========================================================================"
                echo "$log"
                echo "$info"
                echo "$context"
                echo
            fi;
        done;
    done;
fi

/usr/local/bin/svnrevisions

#!/bin/sh
# Usage:  svnrevisions $file
# Output: list of fully numeric svn revisions (without the r), one per line

file="$@"
    svn log "$file" 2> /dev/null | awk '/^r[[:digit:]]+ \|/ { sub(/^r/,"",$1); print  $1 }'

Ajaajaccio

'A' for effort! (just use git :)) – Zavras

Answer 15

In case you are trying to determine which revision is responsible for a specific line of code, you are probably looking for:

svn blame

Credit: original answer

Fornicate

Answer 16

I came across this bash script, but I have not tried it.

Blackbeard