How can I rewrite history so that all files, except the ones I already moved, are in a subdirectory?

Question

I have a project under git. One day I moved all project files from current directory to foo/bar/ under the project. I did it using git mv. Then I added some more files and did some changes to already existing files.

As a result, now when I look at the history of foo/bar/file.c, I can only see changes that I did after I moved the file.

I tried to fix this in various ways (filter-branch with subdirectory filter, etc), but nothing helped, so I am pretty stacked here. I'll appreciate any help that you can give me. Thanks!

Indiscriminate

Answer 1

To rewrite the history with the files moved:

If you want the project's history to look as though all files have always been in the directory foo/bar, then you need to do a little surgery. Use git filter-branch with the "tree filter" to rewrite the commits so that anywhere foo/bar doesn't exist, it is created and all files are moved to it:

git filter-branch --prune-empty --tree-filter '
if [ ! -e foo/bar ]; then
    mkdir -p foo/bar
    git ls-tree --name-only $GIT_COMMIT | xargs -I files mv files foo/bar
fi'

Now the history will be recorded as if all files were always located in foo/bar.

To see the history of a moved file:

If you just want to see the history of a file that has been moved or renamed at some point in the past, then simply use the --follow option to git log:

git log --follow foo/bar/file.c

Tsarism

@Alexander: You may want to try putting the one-liner (everything in the single quotes) in an actual script, with a bash shebang line. I think filter-branch is probably trying to run this in sh, not bash. – Selena

I made some progress. It seems that double brackets in if statement were causing this error. Single brackets work better, it still doesn't work though. The problem is that before I moved everything to foo/bar/ I had directory named foo with some content in the project. So now it complains "cannot move foo' to a subdirectory of itself, foo/bar/foo'". – Indiscriminate

@Alexander: You'll just need to adjust the script accordingly. Something like: if [[ ! -e foo/bar ]]; then mkdir -p foo/bar; git ls-tree --name-only $GIT_COMMIT | grep -v ^foo$ | xargs -I files mv files foo/bar; fi That uses grep to filter out foo from the list of files to be moved. Note that I'm also assuming that the bar subdirectory never existed until you moved everything under it. If that isn't the case, then you'll need to account for that in the script as well. – Tsarism

This works great except for the case when the repo originally contains submodules. The submodule path won't be updated. I'm assuming I can just remove the submodule and re-add it under the new path and add a commit, but that's not quite as elegant. Anything my Googling missed? – Ineffective

Using [[ caused a problem for me, as there is no such command (it is a shell bulitin), using [ and ] solves the problem. – Messenger

This seems to work, however, file that are deleted stick around in the rewritten repository. – Damalas

Can this easily be undone if you are unhappy with the results after doing it on a large repo? – Scorbutic

@Scorbutic yes, it can be easily undone. git filter-branch will keep copies of the original branches under refs/original. So while on branch foo, you can do something like git reset --hard original/foo to put things back the way they were. – Tsarism

Answer 2

To wrap things up here's a short summary of what I did. The command that worked for me was:

if [ ! -e foo/bar ]; then mkdir -p foo/bar; git ls-tree --name-only $GIT_COMMIT | grep -v ^foo$ | xargs -I files mv files foo/bar || echo ""; fi

The echo command that I added at the end ensured that even when mv fails entire command will continue running. It didn't move contents of foo/bar/foo, but I can live with that.

Thanks a lot to Dan Moulding (!!!) and Jefromi for the help.

Indiscriminate

Using || true instead of || echo "" is faster. – Rakish

Answer 3

Install git-filter-repo:

git clone https://github.com/newren/git-filter-repo/

Add the git-filter-repo executable to your $PATH (see INSTALL.MD for other options):

# this is just an example, you probably want to edit bashrc,
# or add a symlink to ~/bin or /usr/local/bin
PATH=$PATH:${PWD}/git-filter-repo

Now cd to the git repo that you want to modify, and run:

git filter-repo --to-subdirectory-filter foo/bar

Astound