What are the options when working with Git submodules from which commits are made?

Question

At work, we're working on a dozen Java OSGi bundles, each of which has its own git repository. All bundles will be, in the long run, pretty independent from each other, which justifies the individual repositories — although right now we're still often modifying several of them at the same time.

When we make a product release (which consists of all bundles), a new branch is created in each bundle, which is a bit of a pain. We were thus thinking about using git-submodule to ease the pain (something like git submodule foreach <cmd>).

So, our desired setup would be a master project Product, and submodules for each bundle:

Project/
  BundleA/
  BundleB/
  BundleC/

Now, I've spent several hours reading all I could find about submodules, and I understood that if I modify things in BundleA, I have to commit in BundleA, push, then commit the submodule change in Project and push again.

This clearly sounds like it was not how git-submodule was designed to be used in the first place. Is it against best practices to use it like this? Or does it sound like a case where an alternative would be preferred?

• bare-bones git-submodule usage
• using an existing “git wrapper”:
  • GitSlave
  • Google repo
• write my own simple bash scripts to batch process the OSGi bundles

Any other suggestion welcome.

Garibold

Answer 1

If you do not need to keep track of what the subproject is doing in the superproject (tight binding) then I recommend you stay far away from git-submodules.

gitslave (http://gitslave.sf.net) is essentially a large foreach around each of the subprojects with a config file in the superproject listing the subprojects. There are lots of bells and whistles which make it more convenient, but if your goal is to run the same command on the superproject (optional) and all of the subprojects, gitslave is about as convenient as you are going to find.

So for example, to create a new branch in gitslave is:

gits checkout -b newbr

Then your superproject and all of the subprojects will create the new branch and change to it. In general, if you want to run a gits command on all members of the superproject, just change the "git" command to "gits".

Doug

So gitslave does not use git-submodule behind the scenes? – Garibold

@Jean-Philippe Pellet: Not at all. With gitslave you have a loose federation of git repositories which allow you to either run git commands individually on each repository, or use gitslave to run your command on all repositories (or a subset). Basically you are trading the forensics diagnostics (being able to say at commit A on the superproject you were at commit B on the subproject) and standardness of git-submodules for simplicity of understanding and use. gitslave provides a loose binding between the repositories—only at tag milestones can you match commits between repos 100% accurately. – Doug

Answer 2

Now, I've spent several hours reading all I could find about submodules, and I understood that if I modify things in BundleA, I have to commit in BundleA, push, then commit the submodule change in Project and push again.

This clearly sounds like it was not how git-submodule was designed to be used in the first place. Is it against best practices to use it like this?

I think that's exactly how submodules are supposed to be used, I'm afraid. Although perhaps the assumption is usually you only commit a new version in the main project relatively infrequently, since you're making a strong assertion that this version of the submodule works properly with that version of the main project. The current design of the submodule system is rather restricted in what it can do, since the only things the main project knows about the submodule are:

• What commit the submodule should be at, stored in the tree instead of a hash of a blob or a tree
• A default URL for a repository that (should) contain that commit, stored in .gitmodules and set in config on submodule initialization

... and when you're working in the submodule, that's like working in a standalone repository, whether there's a super-project there or not.

Of the ways of simplifying this process that you mention, it's worth clarifying that repo actually doesn't use submodules - it's an alternative system. This is similarly the case for git-slave, which many Stack Overflow users seem to recommend.

Personally, I work on a project whose main repository has 24 submodules, and we've done fine with just a helpful script that simplifies the process of committing a new version of the submodule and making sure that it's pushed properly.

Another (non-submodule) alternative that you might want to look at is git-subtree, not to be confused with the subtree merge strategy.

Ichthyology

As Seth noted in the comments to the accepted answer, gitslave does not use git submodules; it tracks the related ("slave") repositories itself. – Exsert

@Alex Dupuy: oops, thanks for pointing out that misunderstanding. I've corrected that now. – Ichthyology