undo git filter-repo and restore deleted repository

Question

I did a git filter-repo and now all files are deleted. VS code gitgraph is showing that the repo has no commits. Is there any way to restore my repo? My whole project got deleted and its a local repo so I don't have any backup. I have been working on it for months. Its killing me. PLEASE HELP. I'm desperate.

I have a commit-map file in the filter-repo folder in the .git with contents like this:

old                                      new
2dcc8b30c4fb13d32f7b448c77a487de1a1d9bd5 0000000000000000000000000000000000000000
4e6915559685c96fe9dc336e1c929f129b603cbc 0000000000000000000000000000000000000000
8bf8d9d278523bf10d50b568eab1f8e5ac6cf8eb 0000000000000000000000000000000000000000
06a897ea4763c3f6034ccdd15e906580dda5f27b 0000000000000000000000000000000000000000
92e8933b223cd7e40bc80f9ac0a2853978bede3d 0000000000000000000000000000000000000000
0b7e2ade7c560078178cb66c0b3acd7d50b5fc10 0000000000000000000000000000000000000000
c296e3f6dd1cfcea8c3ec536c21d17112a9a0222 0000000000000000000000000000000000000000
8ce24d364c43023585109ef680dc2e4dadeeeff6 0000000000000000000000000000000000000000
e9614633a8463e3cdcf0a47206e0639a06aa7527 0000000000000000000000000000000000000000
66f3b8712aa57eaa396251dcc0f5f3285db5ff3b 0000000000000000000000000000000000000000
44442eace317b2b19063cc8c31ffe4a81d216ee7 0000000000000000000000000000000000000000
42877255047bf0376002b98173607a6a89bfab71 0000000000000000000000000000000000000000
54db7f7714302506a137c7b28bcb9a2bad7fc2b4 0000000000000000000000000000000000000000
0f3416c5c8fd3951d7e59a4d23db63f74b9383a3 0000000000000000000000000000000000000000
56379fdd472c102dd50691d9294d27da4b7c9a3d 0000000000000000000000000000000000000000
cc6bb41310efe3e5c2d0f58c956b3dcfd2b9e41a 0000000000000000000000000000000000000000
d7b1b929c943d923e91640b5bcdf49003c6f97cd 0000000000000000000000000000000000000000
f70375aae8340cccfb3c7db235e08464d9cfd1d8 0000000000000000000000000000000000000000
1dfe31b45b82aeb7ef514d81f26547736d60501a 0000000000000000000000000000000000000000
d19e2c4829a2c38c1a0c2adbe60e5a9ca70f7700 0000000000000000000000000000000000000000
7961c20ba12b49ef13996d34f5c4ac6038848418 0000000000000000000000000000000000000000
26ca13021b43c0b8b1b4e6a03948eb091ac6eb12 0000000000000000000000000000000000000000
f03d117ed04f449ae4f4ec1e61e895883acbe491 0000000000000000000000000000000000000000
f0d6ee7e4c778d28f65c864752f0e5cb80648fca 0000000000000000000000000000000000000000
5ce120582976dce9e8921c26adfdd7e5f23814c8 0000000000000000000000000000000000000000
818cd9397dbaa6278f0c33d9ce0ee5115c0c071b 0000000000000000000000000000000000000000
c4cd1317a269f9dc59ded0b9888754c6c205b711 0000000000000000000000000000000000000000

and a ref-map file with:

c4cd1317a269f9dc59ded0b9888754c6c205b711 0000000000000000000000000000000000000000 refs/heads/master

Is there any way I can restore. Its a disaster. I'm literally crying. PLEASE HELP

Edit: commands i used

D:\my_project>git filter-repo --force --subdirectory-filter back\ 
Parsed 27 commits
New history written in 0.53 seconds; now repacking/cleaning...
Repacking your repo and cleaning out old unneeded objects
Updating files: 100% (6161/6161), done.
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Completely finished after 9.45 seconds.

D:\my_project>git status
On branch master

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .idea/
        .vscode/
        front/

nothing added to commit but untracked files present (use "git add" to track)

D:\my_project>git log
fatal: your current branch 'master' does not have any commits yet

Rattray

1) Don't you have a repository at a remote you can pull the old branches (recover) from? 2) Is c4cd1317a269f9dc59ded0b9888754c6c205b711 still reachable, say with git show c4cd1317a269f9dc59ded0b9888754c6c205b711? (If yes and it shows something meaningful, then you can set the master branch back by git branch -f master c4cd1317a269f9dc59ded0b9888754c6c205b711 doing the same for the rest of the branches). 3) What does happen if you do git show-ref? – Calcify

my bad, I don't have a remote repo. git show-ref does nothing. git show c4cd1317a269f9dc59ded0b9888754c6c205b711 returns fatal: bad object c4cd1317a269f9dc59ded0b9888754c6c205b711 – Rattray

It's hard to believe that git show-ref does nothing (otherwise it looks like an empty repo; OR git filter-repo does not retain old refs in a refs namespace similarly to git-filter-branch?). By the way, is git reflog working`? – Calcify

git reflog returns fatal: your current branch 'master' does not have any commits yet – Rattray

git filter-repo requires that you start with a clean clone of some existing repository, and then builds a new repository. Do not overwrite the old repository with the new one! (At least, not yet.) Make sure the new repository is what you want. Archive the old repository somewhere before switching to the new one. The new repository is not compatible with the old repository; do not attempt to use both. – Blaisdell

Do you have the list of commands to replay your case? I just cloned a real repo, ran git-filter-repo with some dummy arguments against it, and I still seem to be able to reconstruct the original repo: a lot of entries in the map files are not set to zeroes (don't know why yet), and git show-ref shows that git-filter-repo created many refs/replaced/... to recover from. (Also its help documentation suggests using --source and --target in order not to break the existing repository...) – Calcify

@Calcify I added the commands i used in the question. – Rattray

I'm not sure if you haven't been running those commands twice to replicate the logs into your question... But, as a last resort, does the content of .git/objects represent something meaningful? (the info and pack directories (with .idx and .pack files inside), possibly a lot of xx directories that probably weren't cleaned up but still containing git objects) -- if so, then they possibly might be used to recover from (or at least to be backed up in order to pick data from), however at very low level. I'm out of ideas. – Calcify

I can't edit my previous comment any longer, but if the .git/objects directory is "big enough", better back up/archive the .git directory and put the backup in a very safe place. Git has some low-level commands listed here: git-scm.com/docs/git#_low_level_commands_plumbing that can be used to peek into the contents of the .git directory internals (but one more time: with caution!) and possibly reconstruct at least some blobs (files). If that directory is not "big" and your 28 commits are really gone... then sorry. – Calcify

here's a link to my .git/objects folder .git/objects . both pack and info have only 2 files in them and the total .git folder is only 52 kb. link to .git folder .git full folder – Rattray

@Calcify Is there no way to get it back? I need an answer so I can start building it again. – Rattray

Sorry to say that, but I guess no, since what you provided as a zip is an empty git repo. – Calcify

for anyone who faces this with flutter... I was able to get most of my flutter code from kernel_blob.bin which is in the apk file built to run the app for testing. At least I have a part of it now and can use it to build the project again. and for the node backend.. unfortunately I completely lost it and will have to do it again. – Rattray

@Calcify I tried recovering files with Recuva software and I probably got some of the .git folder back. It's around 19 mb and has quite a few folders. Could you please look into it and help. I'd be really really grateful. Here's the link to recovered .git folder – Rattray

I cannot tell you much on that unfortuntately. The restored files seem to be broken from git perspective as they usually appear for deleted files that may be (partially) overwritten after deletion. For example, git verify-pack -v cannot parse .git/objects/pack/.....pack reports: bad. The only thing I guess you can do with that is going through each file and checking whether taking the restored content from the bundle would help you recovering your work (I saw some text files there that are clearly JSON or SVG, etc). Sorry, unfortunately this is where my "expertise" and help end... – Calcify

@Calcify very thankful for your time and effort. Just one thing. How to see the svg, json... files like you? is there a git command for that? – Rattray

If you're using Linux, it's easy, in .git/objects, emit find -type f | xargs -I{} file {} to enumerate each file (using find) and try to determine its type by content (using file). Some of the files types are reported as ASCII text, Java source, ASCII text, etc. For compressed stuff reported i.e. as zlib compressed data you can use unix.stackexchange.com/questions/22834/… to decompress the blob (note is compressed data, not an archive). – Calcify

Since you're on Windows, you can also use that set of commands by using git bash (I guess) or Cygwin (that has the core utils ported bundle) to "linuxize" your machine. This would help you to grab as much data as possible. And probably efficiently, if you script it. Also, if you're using file managers (I do prefer Far Manager), you can simply iterate over each file in the .git/objects/XX directories and simply use Quick View panel using Ctrl-Q to detect the blob content by your eyes. It would be tedious, but maybe those hints would let you recover as much as possible. – Calcify

gofile.io/d/rhMxz8 - This is what I could recover using very simple scripts. I don't think you can grab much from it, though, but you can try dig deeper if you like. Good luck! – Calcify

Same happened to me a couple hours ago. I'm shocked... Did you find any solution for recovering the repo? My .git folder has over 1GB still... – Breakthrough

Answer 1

As far as I can tell from the comments of a post that recommends this command and other forum threads, there doesn't seem to be a way to reverse this (Although I am not entirely sure).

Anyway, if you were using an IDE, there is a decent chance that the IDE cached the changes made to your files in a history and that you can restore from there. I know for a fact that the Jetbrains IDEs (PyCharm, IntelliJ, Clion) have a feature like that and it saved my ass yesterday.

To do that in the Jetbrains IDEs you choose "File > Local History > Show History" and restore a snapshot before you ran the command.

This answer will surely be to late for you but maybe i can save the day for someone else...

Viminal

Answer 2

Fetch the original repository before git-filter-repo was called into current repository:

 git fetch . +refs/original/*:*
    

This creates the original version of the repository (eg. my_project) and its commit history in current directory. Set that repository origin to the url of your remote repository.

cd my_project
git log # should have complete git history before git-filter-repo
git remote add origin <remote repository url>

Force push the local repository to update the remote:

git push -f

The remote repository should now have the commit history and files restored. You might have to reclone it locally.

Toffee

Answer 3

As @andiw pointed out, most IDEs have a backup system for unsaved files – this includes VS Code.

For Mac users of VS Code can retrieve their recently changed files in this folder (MacOS Monterey):

~/Library/Application\ Support/Code/User/History

There should be a similar folder for other platforms which you can google.

Additionally, at least for my system, VS Code assigned each recently edited file a folder, with the folder containing snapshots of that file over time. Given the files/folders names are random strings, the folder also contained an "entries.json" which had a "resource" key containing the full path of the file the folder pertains to.

The process I took to recover my files was:

1. Sort folders within the "History" folder by most recent
2. Open latest folder and sort by most recent
3. Determine file name from entries.json and copy/transfer contents of the most recent file to recovery folder or original repo
4. Repeat 2 & 3 until your luck runs out

Obviously, VS Code won't backup all your files, especially if you have a large repo. However, for my situation I was able to recover everything through the arduous process above.

Corded