How to add a git repository as a shared dependency of another git repository?

Question

I need something akin to submodules, but which exist outside the main repository as a dependency.

Here's the problem:

I'm trying to use Git (in a REALLY awkward way) to manage design files for a CAD tool (Cadsoft Eagle), and I'm having a hard time figuring out if there is a way to use git submodules to manage each project's dependency upon the CAD tool's shared library.

I'm using a folder structure like this:

~/eagle/ <-- Main library used by multiple projects
    .git/     
    <library files>

~/projects/ <-- Projects folder
    Proj0/
        .git/
        <design files>
    Proj1/
        .git/
         <design files>

In this case, it doesn't make sense to add the eagle.git repository as a git submodule for each project.

However, I still need a way to snapshot the current state of the "eagle.git" repository so that if the library is updated in the future, it can be rolled back to access the specific revision of the library files which were being used when the Proj[x] was committed.

Ideally, I'd like something like the following:

~/eagle/ <-- Main library used by multiple projects
    .git/     
    <library files>

~/projects/ <-- Projects folder
    Proj0/
        .git/
        <design files>
        **eagle** <-- something that acts like a submodule  
                      but which actually points to ~/eagle/
    Proj1/
        .git/
         <design files>
         **eagle** <-- something that acts like a submodule  
                       but which actually points to ~/eagle/

I'd like to be able to:

cd ~/projects/Proj0
git submodule update

and have the ~/eagle/ directory automatically roll back to the revision checked into Proj0.

Anybody know of anything in Git that could allow for this kind of behavior?

Mile

Can you clarify why submodules won't work for you here? It sounds to me like submodules are exactly what you need. – Parthinia

For the CAD tool (Eagle) to "see" a library, it has to be added to the Eagle's path settings. If I added the "eagle" library repo as a submodule for each project, I'd have to manually append each project's submodule library path Eagle's path settings, which would cause [x] copies of the "eagle" library to show up in Eagle's library manager. Figuring out what's what and managing those separate copies in the CAD tool would be a nightmare. Also, the library can be orders of magnitude larger than the project files, so it seems really wasteful to have [x] copies of it sitting around on disk. – Mile

A helpful way to think of this is via an analogy. Lets say Eagle CAD tool is a factory that can make one widget at a time. Lets say that each project[x].git repo is represented by a manufacturing blueprint[x] for some widget[x] the factory makes. The eagle.git repo is represented by the factory setup (the machines, workers, raw materials) required to make that widget[x]. – Mile

What I need is a way to tag the blueprint[x] with the current factory state required to make that widget[x]. That way, if the factory setup for making widget[x] changes to make widget[y], the factory owner can later use the tag from blueprint[x] to revert the factory back to the EXACT setup required for making widget[x]. – Mile

Having to keep a separate factory for each blueprint doesn't make much sense, but having a single factory that can be reconfigured to make all [x] widgets is necessary. In this case, submodules don't really make sense because the eagle.git repo is not conceptually a component (or sub-anything really) of each project, in the same way a the factory is not a sub-component of a widget. – Mile

However, the eagle.git repo's current state (i.e. revision) is required for the CAD tool to exactly recreate a specific output from a project.git repo, which is why I need to check it into the project.git repos. (Similar to the way different Makefiles can create different binaries from the same source) – Mile

Answer 1

For each project, add .git/hooks/pre-commit (and make sure it's executable):

#!/bin/sh
git --git-dir=~/eagle/.git log -1 --pretty=format:%H >.eagle_rev
git add .eagle_rev

Then, for each project:

git config alias.update-eagle '!git --git-dir=~/eagle/.git --work-tree=~/eagle checkout -q $(<.eagle_rev)'

When you make a commit, it will record the current HEAD of ~/eagle, and git update-eagle will check out that commit in ~/eagle. (Then just make sure you git checkout <branch> in ~/eagle before you make any changes to it.)

Iveson

mkarasek, you are the new hotness! I had no clue about hooks until now... However, for some reason, using "~" in the --git-dir path returned the error "fatal: Not a git repository: '~/eagle/.git'". However, if I replaced "~" with "$HOME" (i.e. '--git-dir=$HOME/eagle/.git') it works fine. Any idea why "~" doesn't work with --git-dir and --work-tree? – Mile

You get that error with the commit hook? Could be that your /bin/sh is having issues with it; replace the first line with #!/bin/bash and see if that fixes the problem. (If, on the other hand, you're getting that error with the alias, then I have no idea.) – Iveson

Answer 2

If eagle hasn't its place within a ProjX,
but each ProjX can use a specific revision of eagle,
then:

For each ProjX, you need to:

• have a MainProjX Git repo, in which you would find:
  • a ProjX
  • a version of eagle (at the same level than ProjX)

The goal of each MainProjX parent project is to keep together the versions of ProjX and eagle, that is to record the right dependencies.

~/projects/ <-- Projects folder
    MainProj0
      Proj0/
          .git/
          <design files>
      eagle/
          .git/
          <library files>

    MainProj1
      Proj1/
          .git/
          <design files>
      eagle/
          .git/
          <library files>

Now, yes, that is a lot of 'eagle' duplication, but that is necessary if each ProjX is able to use its own eagle revision.

Soutache

VonC, thanks for the quick reply. Yeah, that was the only thing I could come up with as well... The problem is that the 'eagle' repo could be really massive (an entire cad library) whereas the Proj[x] repos are really small. Like you said, I'd have to keep around tons of copies of the 'eagle' repos, AND even worse, I'd have to manually change the CAD tool's library path to the individual Proj[x]/eagle/ dir for the tool to even see them, which makes this method practically unusable. – Mile

@lotharsmash: wouldn't a relative path "../eagle" be enough for each library paths? (since eagle would be at the same level than ProjX) – Soutache