How to use git submodules with Bitbucket pipelines?

Question

How to use git submodules with Bitbucket pipelines?

I'm using Bitbucket pipelines to build my project and I'm having issues pulling in my submodule, I'm probably not configuring the SSH keys correctly.

What I've done:

1. Created SSH key pair in my computer.
2. Pasted the same key pair in both repositories (repo where the build will run and in the dependency repo) under Settings/SSH keys.

The build error:

Submodule 'dependencies/my-dependency' ([email protected]:mycompany/my-dependency.git) registered for path 'dependencies/my-dependency'
Cloning into 'dependencies/my-dependency'...
Warning: Permanently added the RSA host key for IP address '18.205.93.2' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Clone of '[email protected]:mycompany/my-dependency.git' into submodule path 'dependencies/my-dependency' failed

My yml file

image:
  name: myuser/my-image-name
  username: $DOCKER_HUB_USERNAME
  password: $DOCKER_HUB_PASSWORD
  email: $DOCKER_HUB_EMAIL

pipelines:
  branches:
    pipelines-setup:
      - step:
          script:
            - git submodule update --init

Zakaria

Answer 1

Found the solution. I had to add the ssh public key to the submodule repository under Settings / Access Keys not Settings / SSH Keys.

Zakaria

Answer 2

1. Your source repository should contains file MyProject/.gitmodules with paths to submodules (the paths to the repos in you .gitmodules file should be in the [email protected]:....git format):

[submodule "modules"]
    path = modules
    url = [email protected]:....git

[submodule "translations"]
    path = translations
    url = [email protected]:....git

• Open repository where do you want to run pipelines
• Open Settings
• In section PIPELINES open SSH keys
• Click on Generate key
• Copy public key

now you need to add the ssh key to submodule repository

• Open submodule repository
• Open Settings
• In section SECURITY open Access keys
• Add copied ssh public key

Holley

In my setup, I don't have access to the submodule repo's "settings" (I only have read access) so I had to add the pipeline key to my personal SSH keys - and that worked, though probably not ideal? – Argumentative

Want to emphasize that the paths to the repos in you .gitmodules file should be in the [email protected]:....git format. – Optimism

"In section GENERAL open Access keys" now it's under SECURITY section – Agenesis

Answer 3

here is another example that

• uses the default image
• adds submodules
• zips only the needed files and uploads to bitbucket downloads

---

image: atlassian/default-image:2
pipelines:
  default:
    - step:
        deployment: production
        script:
          - git submodule update --recursive --init
          - apt-get update && apt-get install -y zip
          - zip -r Test.zip . -x bitbucket-pipelines.yml *.git*
          - pipe: atlassian/bitbucket-upload-file:0.1.3
            variables:
              BITBUCKET_USERNAME: $BITBUCKET_USERNAME
              BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
              FILENAME: Test.zip

Mcentire

Answer 4

If you have initialized a submodule in the main repository, then you can add this command in the bitbucket pipeline step script:

- git submodule update --init --recursive

No need to use SSH Keys approach.

Kernite

Wrong. Without access to the submodule the pipeline will not be able to read it. You can get access easiest with ssh keys, but you can also publish your password in the yml if you want to publish all your secrets to all the hackers. – Nemertean