How to maintain (mostly) parallel branches with only a few difference

Question

Scenario: I'm trying to get my unix dot-files under git. I have to work between (at least) the cygwin environment and some standard linux distros (ubuntu and opensuse), and I have files/lines of code that are only specific to, say, cygwin. Since I don't want to checkout useless files or have to deal with lots of cases inside my dotfiles, I'm creating branches for each of my environments. But most of the edits I do are common to all environments, so almost every time I made a commit I need to propagate that change to all my branches.

So basically I have several branches that are almost identical except for a few commits, and most commits I do need to be in all branches.

The question: what is the recommended git workflow for this, if there is any? Or is there a better setup (without using multiple branches?) for my scenario?

[I tried cherry-picking, but that involves quite a bit of work, and not to mention all the duplicate commits out here and the nightmare it is to keep my branches in sync.]

Mediacy

I was writing up an answer based on submodules when I discovered someone else beat me to it: #3590900 Basically, make your common files a single repo, and all your environments each their own repo. Then bring the common files in as a submodule. You'll need to re-arrange directory levels though. – Emory

Answer 1

For that particular case, where there is a lot of common files evolving in one branch, and only a few config files specific per environment... we do not store the config file in Git. At all.

We do store template of said config files, plus all the specific per-environment values, plus a script able to replace the variables in the template files by the correct value (detecting the current platform)

That way, we do not need to make a branch only for those files.

---

Another good way to manage those kind of files (with a platform-specifc content) is through a git attribute filter driver (see also Pro Git book).

A filter driver consists of a clean command and a smudge command, either of which can be left unspecified.
Upon checkout, when the smudge command is specified, the command is fed the blob object from its standard input, and its standard output is used to update the worktree file.
Similarly, the clean command is used to convert the contents of worktree file upon check-in.

That way, a script (managed with Git) referenced by the smudge can replace all the variables by platform-specific values, while the clean script will restore its content to an untouched config file.

The main idea remains: avoid creating branches only for that kind of parallel evolution.

Papert

We do create branches for other reasons, of course: see stackoverflow.com/questions/2100829#2107672 – Papert

What did you use to create this image? – Gilbertogilbertson

@Gilbertogilbertson it comes from the git book: git-scm.com/book/en/Customizing-Git-Git-Attributes – Papert

Does anyone have an example of dotfiles managed in this way? This is still fairly abstract. – Lotti

@BretComnes https://mcmap.net/q/12999/-is-it-possible-to-include-a-file-in-your-gitconfig was slightly more concrete but not exactly related to this question. – Papert

@Papert Thanks. Reading that now. Still, it would be fantastic to see an actual working example of a git repo used for dotfiles that takes advantage of git smudge filters. Someone post a link! :D – Lotti

@Lotti not dotfiles, but I do replace some code by another with github.com/VonC/compileEverything/blob/master/gitolite/…, a smudge script declare in github.com/VonC/compileEverything/blob/master/gitolite/…, and put in place with github.com/VonC/compileEverything/blob/master/gitolite/… – Papert

Answer 2

One approach is to keep a branch for each environment, plus a "master" branch that is common to all environments. Whenever you make a change to the master branch and want to pull it into another system, do something like:

git pull
git checkout local
git rebase master

This will rewrite the current changes on "local" (that are for this particular environment) against the current state of "master".

The manual thing you'd need to pay attention to is where you commit a change you want to make. If it's local to a system, commit it to that system's "local" branch, else commit it to "master" and push it up to a common repository.

Of course, rebasing may result in conflicts that you have to resolve manually. Also, if you choose to push local branches to the common repository, you'd have to (a) choose unique names for each environment, and (b) deal with the non-fast-forward pushes after rebasing. Both these problems are solvable.

Atmospherics

Answer 3

Good question. Even though you said:

...Since I don't want to checkout useless files...

I would go for putting the platform-specific or variant-specific items in the same branch as the main code, but in a separate directory for that platform/variant. The key is to isolate the platform-specific stuff to as small an area as possible (i.e. avoid the ifdefs in the main code).

E.g.:

/
+--common
+--linux
+--cygwin
+--windows
+--mac

Various cross-platform projects organise themselves in this way. E.g. check out Python's source code structure for supporting multiple platforms.

It simplifies your workflow in this regard, so you are more free to use branches for other more interesting purposes.

Margiemargin