How can I move a single directory from a git repository to a new repository whilst maintaining the history?

Question

I have inherited a git repository containing multiple projects in separate directories. I'd like to split the repository into new individual repositories, one for each project and then have the master repository contain the projects as submodules. I'd like to do all this whilst maintaining the revision history of the individual projects if possible.

I could clone the repository for each project and remove all the other projects each time, but it there a better way to avoid having the cloned history in each new project repository?

Mittel

Added git-submodules tag since this is very useful for converting part of a repo into a submodule. – Raskind

Possible duplicate of Export subtree in Git with history – Sangraal

Answer 1

You can use git filter-branch to rewrite the history of a project. From the documentation:

To rewrite the repository to look as if foodir/ had been its project root, and discard all other history:

git filter-branch --subdirectory-filter foodir -- --all

Make several copies of your repo, do that for each subdirectory you want to split out, and you should wind up with what you're looking for.

Phocis

by doing this, does this start the new repo as if there were no history whatsoever then? that's what I am gathering from the doc. – Sheaff

@dtan The new repo will have all of the history that affects the foodir/ directory, but none of the history that affects anything else in the repo. So if you had four subdirectories, each of which was separate projects, and ran git filter-branch on each one, each would have only the history that affected that subdirectory. – Phocis

What if I wanted to exclude foodir, and remove all it's history? – Kenosis

Note: if you want several directories, You will need to specify --subdirectory-filter multiple times. E.G. git filter-branch --subdirectory-filter foodir --subdirectory-filter bardir, etc. --subdirectory won't take multiple dirs, but can be specified multiple times. – Lonne

@BrianCampbell How do I clean up the original repository after running this command? I just want to move one folder out into its own repository. Afterwards I want to remove it from the original repository. Will git rm -r --cached foodir work? – Airtoair

@Airtoair If you want to keep the history of foodir in the original project, not rewriting its history, just git rm -r foodir should be sufficient (that will also delete the copy in your working tree; if you don't want that, use --cached). If you wanted to remove it entirely from the history (answering @ilius's question too), you want something like git filter-branch --index-filter 'git rm -r --cached --ignore-unmatched foodir' -- --all – Phocis

@ilius Sorry I missed your question earlier, see my reply to above for an answer for how to remove a directory and its history. – Phocis

Thanks @BrianCampbell, I found this command but forgot to post it: git filter-branch --tree-filter 'rm -rf foodir' HEAD – Kenosis

@ilius Yep, that works too. For a large project, the --index-filter solution is faster than the --tree-filter, as it doesn't have to actually check out the files, it can just manipulate the index directly. The --tree-filter can be a little easier to use however, as you can use ordinary filsystem operations rather than having to work with index manipulation operations. – Phocis

@ilius Just one more comment: if you don't have all the "remote branches" checked out locally, the tree-filter will not be applied to them. – Upu

On top of that, if you also want to remove "empty" commits after removing the history for some files using --tree-filter, you'll have to use --prune-empty too – Upu

Answer 2

To export a folder as a new repository you need:

1. To clone the repository where the folder you want to export is.
2. To create a empty repository on your hosting provider as GitHub, to store the exported folder.

3. Open the cloned repository folder and run this command:

   git subtree push --prefix=YourFolderNameToExport https://github.com/YourUserName/YourNewCleanRepoName master
   

Sangraal

git subtree is not available as Cygwin package. If you need it: https://mcmap.net/q/196283/-how-do-i-install-git-subtree-with-cygwin-in-windows-7 – Template

Answer 3

The point of git is that the history is embodied in each commit by hashing the parent commit. You could "replay" the commits (this is essentially how the svn-importer works) into a new repository and only keeping each sub-project. This, however, would destroy the meaning of the commit hashes. If you have no problem with that then so be it.

In the past I've just cloned it and moved on. This makes things larger but disk space is cheap; my time is expensive.

I also don't know of any tools to splice out a directory. I suppose you could git-log on the directory to find all commits on it then replay the commits with something like git-fast-export?

Teflon

I upvoted because this because didn't deserve to be negative -- I certainly wouldn't follow this approach but can see he was trying – Kearse