How to clone public submodule in "Github Actions"

Question

Reason for this Q&A-Style question: It took me a few hours to get this to run because I had some typos and thought the solution is more complicated. If I would have found a tutorial like this on google or Stackoverflow I would have checked for typos.

Git Repository Setup:

• Private repository A - name: repoA
• with submodule B (public repository) - name: repoB

Goal:

• I want to run a gradle build in repository A > Github Actions

Github Action Workflow

name: Test
on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1

Problem:

• The actions/checkout@v1 step fails to access the submodule

.gitmodules

[submodule "library"]
    path = library
    url = [email protected]:organization/repoB.git

Github Actions Step Build with Gradle error

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':repoA:compileReleaseAidl'.
> Could not resolve all task dependencies for configuration ':repoA:releaseCompileClasspath'.
   > Could not resolve project :repoBSubmodule1.
     Required by:
         project :repoA

What I tried:

---

1. Add with: submodules: true to actions/checkout@v1

    - uses: actions/checkout@v1
      with: 
        submodules: true

Github Actions Step Run actions/checkout@v1 error

(...)
git submodule sync
git -c http.https://github.com.extraheader="AUTHORIZATION: basic ***" submodule update --init --force
Submodule 'repoB' ([email protected]:organization/repoB.git) registered for path 'repoB'
Cloning into '/home/runner/work/repoA/repoA/repoB'...
Host key verification failed.
##[error]fatal: Could not read from remote repository.

---

2. Use 3rd party Github Actions like textbook/[email protected]

Run textbook/[email protected] error:

fatal: Not a git repository (or any parent up to mount point /github/workspace)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
##[error]Docker run failed with exit code 128

---

3. Add a personal access token to the actions/checkout

   • The token was generated from a github user which has access to that repository
   • The repoA is owned by an organization
   • the token has full repo permissions

    - uses: actions/checkout@v1
      with: 
        submodules: true
        token: ${{ secrets.GITHUB_REPO_TOKEN }}

Run actions/checkout@v1 error:

git submodule sync
git -c http.https://github.com.extraheader="AUTHORIZATION: basic ***" submodule update --init --force
Submodule 'repoB' ([email protected]:organization/repoB.git) registered for path 'repoB'
Cloning into '/home/runner/work/repoA/repoA/repoB'...
Host key verification failed.
##[error]fatal: Could not read from remote repository.

I.e. with the that token which has access to both, repoA and repoB I was not even able to checkout the parent repoA.

Job

Answer 1

Now you can use actions/checkout@v2

As https://github.com/actions/checkout#checkout-submodules said:

- uses: actions/checkout@v2 # checkout root
- name: Checkout submodules # checkout rest
  shell: bash
  run: |
    # If your submodules are configured to use SSH instead of HTTPS please uncomment the following line
    # git config --global url."https://github.com/".insteadOf "[email protected]:"
    auth_header="$(git config --local --get http.https://github.com/.extraheader)"
    git submodule sync --recursive
    git -c "http.extraheader=$auth_header" -c protocol.version=2 submodule update --init --force --recursive --depth=1

Greater

Should there be a - at the start of both top lines here? Trying to get this to work on windows. – Vern

@Vern Sorry, my mistake. These are two steps and each step needs a - – Fatten

That action can do submodules without additional scripting. – Sangraal

Answer 2

You can use actions/checkout@v2 without additional scripting. I used this on Mac, Linux and Windows. Meanwhile v3 is available:

- uses: actions/checkout@v3
  with:
    submodules: true

For recursive submodules (where a submodule requires another submodule), use

- uses: actions/checkout@v3
  with:
    submodules: recursive

Sangraal

Answer 3

Changing the submodule URL from the SSH to the HTTPS format fixed it:

.gitmodules

[submodule "repoB"]
    path = repoB
#Before:
    url = [email protected]:organization/repoB.git
# After:
    url = https://github.com/organization/repoB.git 

Job

Answer 4

For Windows based runner the following worked: Created a bot-user, generated Access Token, included the user into private (submodules) repos and added this in actions.yml:

  - name: Checkout submodules using a PAT
    shell: bash
    run: |
      git config --file .gitmodules --get-regexp url | while read url; do
      git config --file=.gitmodules $(echo "$url" | sed -E "s/[email protected]:|https:\/\/github.com\//https:\/\/${{ secrets.GHACTIONS_PAT }}:${{ secrets.GHACTIONS_PAT }}@github.com\//")
      done
      git submodule sync
      git submodule update --init --recursive

Unpolitic

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Nickolenicks

This block alone just gave me authentication errors like fatal: Cannot prompt because user interactivity has been disabled. bash: /dev/tty: No such device or address error: failed to execute prompt script (exit code 1) fatal: could not read Username for 'https://github.com': No such file or directory – Sangraal

@Hiran Chaudhuri Not sure why this happens for you, have you provided the personal access token (PAT) in you repository secrets as I did with GHACTIONS_PAT ? – Unpolitic

I resolved it by using actions/checkout@v2 with submodules: true – Sangraal