Move file and directory into a sub-directory along with commit history

Question

How can I move a directory and files to a sub-directory along with commit history?

For example:

• Source directory structure: [project]/x/[files & sub-dirs]

• Target directory structure: [project]/x/p/q/[files & sub-dirs]

Sackman

I don't understand most of the answers to this question, I tried all ways I always either got an error, or simply nothing happened, so I just cut the .git folder and pasted it in a sub folder... guess what? It worked! – Truncheon

See related/duplicate question https://mcmap.net/q/121196/-move-all-files-of-a-git-directory-into-subdirectory-and-maintain-history/421049. My answer https://mcmap.net/q/121196/-move-all-files-of-a-git-directory-into-subdirectory-and-maintain-history there covers Windows. – Tympanist

Does anyone know how to to it the other way around. Having a repro in a subfolder and then add some parent files/folder to this repro? – Araldo

Answer 1

To add to bmargulies's comment, the complete sequence is:

mkdir -p x/p/q      # make sure the parent directories exist first
git mv x/* x/p/q    # move folder, with history preserved
git commit -m "changed the foldername x into x/p/q"

Try it first to see a preview of the move:

git mv -n x/* x/p/q

---

Wolfgang comments:

If you're using bash, you can avoid the issue of trying to move a folder into itself by using an extended glob like so (using the shopt built-in):

shopt -s extglob; git mv !(folder) folder

---

Captain Man reports in the comments having to do:

mkdir temp 
git mv x/* temp
mkdir -p x/p/q
git mv temp x/p/q
rmdir temp;

Context:

I am on Windows with Cygwin.
I just realized I did the shopt -s extglob example wrong so my way may not have be necessary, but I typically do use zsh instead of bash, and it didn't have the command shopt -s extglob (though I'm sure there is an alternative), so this approach should work across shells (subbing in your shell's mkdir and rmdir if it's especially foreign)

---

As an alternative, spanky mentions in the comments the -k option of git mv:

Skip move or rename actions which would lead to an error condition.

git mv -k * target/

That would avoid the "can not move directory into itself" error.

Investigate

You can't move x into a subdirectory of itself. git mv ... does the job, no need to git add ... anything if there aren't further changes. – Surtout

How use git mv here? It complains saying I cannot move a directory into a subdirectory of itself. – Jinnyjinrikisha

@Abhinav did it move everything else though? The complain was about x/p which indeed cannot be moved in x/p/q. – Investigate

@Investigate Everything else did move. I was not comfortable using this approach because I was writing a generic script for it. Therefore I ended up moving everything to x/.p/q and from there to x/p/q. That worked. – Jinnyjinrikisha

If you're using bash, you can avoid the issue of trying to move a folder into itself by using an extended glob like so: shopt -s extglob; git mv !(folder) folder – Salcedo

@Salcedo Thank you. I have included your comment in the answer for more visibility. – Investigate

git mv !(folder) folder doesn't work if you have any ignored files. Any workaround for this? – Selfstarter

@mic not on the top of my head. That could be a good question on its own. – Investigate

I had to do this: mkdir temp; git mv x/* temp; mkdir -p x/p/q; git mv temp x/p/q; rmdir temp; – Inhume

@CaptainMan OK. I have included your comment in the answer for more visibility. What OS and what git version were you using for that. – Investigate

@Investigate I am on Windows with Cygwin. I just realized I did the shopt -s extglob example wrong so my way may not have be necessary, but I typically do use zsh instead of bash, and it didn't have the command shopt -s extglob (though I'm sure there is an alternative), so this approach should work across shells (subbing in your shell's mkdir and rmdir if it's especially foreign). – Inhume

@CaptainMan OK. I have added your last comment as it is an important context for your method. I personnally don't use Cygwin anymore, since the new msys2-based bash (https://mcmap.net/q/12302/-why-is-it-that-if-you-download-git-2-0-from-the-net-you-always-get-a-1-9-4-installer-package, and https://mcmap.net/q/12113/-how-are-msys-msys2-and-msysgit-related-to-each-other) – Investigate

I just learned you can use git mv -k * target/ to avoid the "can not move directory into itself" error. – Revis

@Revis Interesting. I have included your comment in the answer for more visibility. – Investigate

I'm not a fan of git mv; preserving the history depends on git's ability to detect rename/move. git log -C usually works, but not always, and you have to remember to use it. – Richellericher

@EdwardFalk I agree. Torek illustrated the "hack" a long time ago: https://mcmap.net/q/121197/-moving-file-s-and-it-39-s-history-to-new-branch. I would move and commit files without modify them, and then only commit new modifications, to improve the chance of git log --follow to actually work. – Investigate

Answer 2

Git does a very good job to track content even if it is moved around, so git mv is clearly the way to go if you move files because they used to belong in x, but now they belong in x/p/q because the project evolved that way.

Sometimes, however, there is a reason to move files to a subdirectory throughout the history of a project. For example if the files have been put somewhere by mistake and some more commits have been made since, but the intermittent commits don't even make sense with the files in the wrong place. If all that happened on a local branch, we want to clean up the mess before pushing.

The question states "along with commit history", which I would interpret as rewriting the history in exactly that way. This can be done with

git filter-branch --tree-filter "cd x; mkdir -p p/q; mv [files & sub-dirs] p/q" HEAD

The files then appear in the p/q subdirectory throughout the history.

The tree filter is well suited for small projects, its advantage is that the command is simple and easy to understand. For large projects this solution does not scale very well, if performance matters then consider to use an index filter as outlined in this answer.

Please note that the result should not be pushed to a public server if the rewrite touches commits which were already pushed before.

Perambulate

thank you, made my day! I wanted to "shift" all files of a tree into a new subdirectory and used your approach with mv * ./the-subdir but got the error: "cannot move the-subdir to a subdirectory of itself. My solution is to use mv the-subdir /tmp; mv * /tmp/the-subdir mv /tmp/the-subdir ./ inside the tree-filter command. Works like a charm. – Latrell

I couldn't get this to work unless I git mv [files & sub-dirs] p/q instead of calling a raw mv; Running on a mac. But it did end up working well for me so, thanks! – Ancheta

I used mv * .* the/subdir/ || true to move files and dotfiles and ignore the error messages. – Lordan

Answer 3

I can see this is an old question, but I still feel obliged to answer With my current solution to the problem, that I have derived from one of the examples in the git book. In stead of using an inefficient --tree-filter I move the files directly on the index With an --index-filter.

git filter-branch -f --index-filter 'PATHS=`git ls-files -s | sed "s/^<old_location>/<new_location>/"`;GIT_INDEX_FILE=$GIT_INDEX_FILE.new; echo -n "$PATHS" | git update-index --index-info && if [ -e "$GIT_INDEX_FILE.new" ]; then mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"; fi' -- --all

this is a specialization of one of the examples from the book I've also used the same example to mass rename files in the commit history in one special case. if moving files in subdirectories: remember to escape the / character in the paths With a \ to make the sed command work as expected.

Example:

Project Directory
                 |-a
                 |  |-a1.txt
                 |  |-b
                 |  |  |-b1.txt

to move the b directory to the project root:

git filter-branch -f --index-filter 'PATHS=`git ls-files -s | sed "s/a\/b\//b\//"`;GIT_INDEX_FILE=$GIT_INDEX_FILE.new; echo -n "$PATHS" | git update-index --index-info && if [ -e "$GIT_INDEX_FILE.new" ]; then mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"; fi' -- --all

Result:

Project Directory
                 |-a
                 |  |-a1.txt
                 |
                 |-b
                 |  |-b1.txt

Adversaria

With you command line, I got fatal: malformed index info -n 100644. With git filter-branch -f --index-filter 'git ls-files -s | sed "s/<old_location>/<new_location>/" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD it worked in my case. Don't forget to escape / with \/. – Pisces

@Pisces If you use : as your delimiter for sed, then you don't have to worry about escaping /. – Larentia

To add another example, if you want to move all your files/folders in the repository root . inside ./newfolder1/newfolder2/ you will have to type: git filter-branch -f --index-filter 'git ls-files -s | sed "s:\t\"*:&newfolder1/newfolder2/:" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD. Hope this helps others! – Breve

With the second example that moves "b" to the root, I get this error after a while: fatal: Unable to create 'C:/stuff/test_rewrite2/.git-rewrite/t/../index.new.new.new.new.new.new.new.new.new.new.lock': Filename too long I use git bash from git for windows version 2.41.0 and the exact command from your answer – Assembler

Answer 4

The git mv command is the simplest way. However, at least on my Linux box, I needed to supply the -k flag to avoid getting an error back that a folder cannot be moved into itself. I was able to perform the action by using...

mkdir subdirectory
git mv -k ./* ./subdirectory
# check to make sure everything moved (see below)
git commit

As a warning, this will skip all moves which would lead to an error condition, so you will probably want to check that everything worked properly after the move and before a commit.

Camelopardalis

Note: -k flag does not work on Mac OS X (Sierra), nor, apparently, on Ubuntu. – Bloody

Answer 5

Here is an alternative way if you for some reason do not want to use the git mv command:

• Make sure you do not have any uncommitted changes.

  git status

• Just move the folder that contains the .git folder to the new folder:

  mkdir new_folder mv old_folder new_folder

• Then move the .git old_folder from the moved folder back to the base folder:

  mv new_folder/old_folder/.git new_folder/

• Then stage and commit all the changes detected (listed as adding files to their new location and removing them from their old location)

Registration

Answer 6

My own suggestion: git fast-export then edit the export file, then git fast-import.

Richellericher