What is the best way to implement git flow with a project with many submodules

Question

We have build an Asp.net core micro-services project and organized the code into one super project and multiple Git submodules (each microservice is a submodule). Now we want to start using the Git Flow workflow.

What is the best way to initlize the gitflow? do we need to have one git flow per submodule OR we should have one global git flow at the super project level ?

Thanks!

Doxology

This may sounds unhelpful but it is meant well: DONT USE GIT SUBMODULES. Too long to explain but basically yeah workflow and refreshing becames a major pain. Sorry but many have been burned by them. Seek an other way to organize your code. – Damiandamiani

I finally ended up removing the sub-modules as they generated more issues than benefits – Doxology

@NabilKem I have included your comment in the answer for more visibility. – Ordain

Answer 1

A project like readium/readium-js-viewer (using submodules) considered using git flow and discussed it in issues 392.

Using git flow would involve decomposing each command into their Git equivalent, used as:

git submodule foreach —recursive 'git checkout develop'

But (in this document):

However, this workflow has turned out to be somewhat problematic for several reasons.

• The git-flow tools (which are essentially bash scripts calling command line git routines) have a number of bugs.
  Most importantly, if there are any problems with the merge or release process, the bash scripts simply fail silently - appearing to work when in fact they do not and often report success when this is not true.
• Git, in general, is somewhat fragile with respect to the use of submodules. For example, if the structure of a submodule changes (e.g. the folder structure changes) then merges and new branch creation can fail as git doesn’t know how to correctly delete obsolete branch fragments.
  The abandoned fragments need to be deleted manually.
• The git-flow workflow seems to be overkill for our relatively small project (compared to, for example, Adobe’s Creative Suite, Eclipse and other large projects).
  The general purpose of a release branch workflow is to test and merge a complex project. If problems are found they can be fixed and the result pushed back to develop.
  In practice, we rarely encounter problems in the RC branch sufficiently severe to resolve in place and re-merge. Instead, we simply log an issue and plan to fix it in the next release.

So the use of the full git-flow workflow - both the tools and the workflow itself - do not seem the best fit for Readium.
In consequence, we propose that Readium adopt a workflow similar to git-flow, but simplified to suit our need

So using branches can be done with git submodule foreach —recursive, ... but you might want to keep said branching workflow as simple as possible.

---

Alternatively, the OP Nabil Kem added in a comment:

I finally ended up removing the sub-modules as they generated more issues than benefits.

Ordain

Answer 2

Submodules are needed to atomically control permissions to folders under git. As of now, I don't know of any viable alternative, hence had to make it work with git flow

Git flow with submodules:

• Initialize/checkout feature branch

BRANCH_NAME=feature/some-feature-name
bash git-checkout-with-submodules.sh "${BRANCH_NAME}"

• Do commits in your IDE(like Rider from JetBrains or VisualStudio from Microsoft)
• Push feature branch

bash git-push-with-submodules.sh

• Merge branch main into current branch

BRANCH_NAME=main
bash git-merge-with-submodules.sh "${BRANCH_NAME}"

• Create PR in your git provider(like github), code review will contain only changed submodules, and will look like on screenshot (Please upvote this discussion which proposes to support atomic merges in PR-s in github https://github.com/github/feedback/discussions/10968)
• After PR is merged into main branch and feature branch is no longer needed, delete branch with

BRANCH_NAME=feature/some-feature-name
bash git-delete-branch-with-submodules.sh "${BRANCH_NAME}"

Scripts

• git-checkout-with-submodules.sh

BRANCH_NAME="${1:-main}"

COMMAND="git fetch && (git checkout \"${BRANCH_NAME}\" || git checkout -b \"${BRANCH_NAME}\")"
# Running command in parent repository
eval "$COMMAND"
# Running command in submodules
bash git-submodule-foreach-parallel.sh "$COMMAND"

• git-clean-with-submodules.sh

COMMAND="git clean -dfx -e \"**/.idea\""
# Running command in parent repository
eval "$COMMAND"
# Running command in submodules
bash git-submodule-foreach-parallel.sh "$COMMAND"

• git-delete-branch-with-submodules.sh

BRANCH_NAME="${1}"
REMOTE="${2:-origin}"

COMMAND="git fetch && (git push -d ${REMOTE} ${BRANCH_NAME}; git branch -d ${BRANCH_NAME})"
# Running command in parent repository
eval "$COMMAND"
# Running command in submodules
bash git-submodule-foreach-parallel.sh "$COMMAND"

• git-pull-with-submodules.sh

COMMAND="git fetch && git pull"
# Running command in parent repository
eval "$COMMAND"
# Running command in submodules
bash git-submodule-foreach-parallel.sh "$COMMAND"

• git-push-with-submodules.sh

REMOTE="${1:-origin}"

COMMAND="git fetch && git push --set-upstream ${REMOTE} HEAD"
# Running command in parent repository
eval "$COMMAND"
# Running command in submodules
bash git-submodule-foreach-parallel.sh "$COMMAND"

• git-merge-with-submodules.sh

BRANCH_NAME="${1:-main}"

COMMAND="git merge \"${BRANCH_NAME}\""
# Running command in parent repository
eval "$COMMAND"
# Running command in submodules
bash git-submodule-foreach-parallel.sh "$COMMAND"

• git-submodule-foreach-parallel.sh

#!/bin/bash

if [ -z "$1" ]; then
    echo "Missing Command" >&2
    exit 1
fi

COMMAND="$@"

IFS=$'\n'
for DIR in $(git submodule foreach --recursive -q sh -c pwd); do
    printf "\n\nStarted running command \"${COMMAND}\" in directory \"${DIR}\"\n\n" \
    && \
    cd "$DIR" \
    && \
    eval "$COMMAND" \
    && \
    printf "\nFinished running command \"${COMMAND}\" in directory \"${DIR}\"\n\n" \
    &
done
wait

Burberry