Inspect git repo after using BFG repo-cleaner

Question

Very basic git question:

I uploaded some compromising information to Github and am using bfg to clean the repo. I followed the documentation and performed the following actions:

$ git clone --mirror git://example.com/some-big-repo.git
$ bfg --replace-text passwords.txt  my-repo.git

I received the following output:

Found 233 objects to protect
Found 9 commit-pointing refs : HEAD, refs/heads/experimental, refs/heads/master, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

 * commit 497fc1c8 (protected by 'HEAD')

Cleaning
--------

Found 80 commits
Cleaning commits:       100% (80/80)
Cleaning commits completed in 301 ms.

BFG aborting: No refs to update - no dirty commits found??

I'd like to see if the private information was cleared from my repo but I'm not sure how to check the files in the mirrored repo. Any ideas?

Seigler

Answer 1

A quick way to check if a password is still in your history might be to use the 'git pickaxe', aka the -S option. Here's an example that checks for the string password1:

git log -Spassword1

However, from the output shown in your question, it looks like The BFG couldn't find any of the entries from passwords.txt in your repo (prompting the messsage 'no dirty commits found??' which you see at the end of the output), which is a bit strange if you're sure they're in there. Was this the first time you'd run the BFG on the repo? Perhaps it was the second time, and The BFG had already removed the passwords?

The passwords.txt file you give to The BFG should have one password per line, ie:

changeme
password1
password2

The BFG only looks at text files under 1MB by default. Are your passwords in some file that might appear to be binary, or bigger than 1MB?

Update: For seeing what's changed in a repo-clean-up, you could also try Eric S. Raymond's repodiffer (part of his reposurgeon project): http://www.catb.org/~esr/reposurgeon/repodiffer.html - you use it like this:

$ repodiffer old-repo-copy.git new-repo-copy.git

The script may take a while to run, but it will tell you precisely what has changed between those two repos.

Full disclosure: I'm the author of the BFG Repo-Cleaner.

Illegible

Thanks. The pickaxe is exactly what I was looking for. I ultimately just decided to use the standard brute force git-filter-branch method to wipe the log history of that seeds file altogether (since it should have been in my .gitignore in the first place). – Seigler

Sure - you can also use bfg --delete-files seeds my-repo.git if you want to delete any file named seeds completely. – Illegible

Ahh, very nice. Thanks again. – Seigler

Hey there, it seems repodiffer isn't a part of reposurgeon anymore. The news document on reposurgeons git mentions "repodiffer has been retired". Would you know of a alternative? – Ad

Answer 2

To double-check if specific sensitive data is not present in your repo anymore, you may use the following command:

git log --patch | grep --color=auto secret

or use less +/secret instead of grep to see it in context.

Kwan

Answer 3

I wanted to leave an addition here, because I spent an hour trying to work out why I was getting the same error message:

BFG aborting: No refs to update

And I absolutely couldn't work out why.

And then I realised that BFG, like git, is case sensitive - and the file name I was typing in was all lower case, while the version in git had an upper case initial letter.

As I was on Windows, I hadn't thought about this - the file system there is not case sensitive.

Hopefully this will save someone else a couple of hours of pain!

Ljubljana

Answer 4

Had the same issue for 2 hours on OSX. In my case the issues was the password.txt file (maybe corrupt or something). Make sure you create the .txt via terminal by using vi/nano. Don't create it with tools like TextEdit and change extension.

Paphian