Jenkins: Retrieving submodules with Git

Question

Currently I've been stuck on an issue trying to retrieve the submodules of a repository from within Jenkins. My configuration is fine and I can pull repositories without any submodules just fine.

I can also pull the main components of a repo with submodules (both with authentication in the repository-name as with SSH). The problem only arises when I have to pull the submodule-components. I'm running the latest version of Jenkins and I've added a part at the bottom which is for "Advanced sub-modules behaviours". I selected "Recursively update submodules" here and ran the build several times to no avail.

When I try adding an extra build step at the bottom with shell-commands, the updating of the repositories doesn't work either. When I try these commands outside of jenkins in my terminal, this works just fine. The issue I always get in Jenkins is:

FATAL: Command "git submodule update" returned status code 1:

stdout: 

stderr: Cloning into 'thisismysubmodule'...
fatal: Authentication failed for 'https://git.thisismyrepo.com/scm/ap/thisismysubmodule.git/'

I've found this issue: https://issues.jenkins-ci.org/browse/JENKINS-20941 but I can't use the suggested solution at the bottom due to security concerns. Does anyone here have any experience with this problem or a possible solution?

Tubman

Answer 1

Here is a workaround using SSH agent forwarding. It worked fine for me.

1. First, edit <jenkins_home>/.ssh/config and set ForwardAgent yes
2. Then, install SSH Agent plugin for Jenkins.
3. Then, in project configuration, reset the Git credentials.
4. Finally, in project configuration, set the SSH Agent credential.

Brunildabruning

And don't forget to do a test clone via ssh from the jenkins' user. Because that would add the remote server to the local known_hosts. – Kalliekallista

See #22050277 as well. The keys must be in the default files .ssh/id_rsa and .ssh/id_rsa.pub or the git submodule update will still fail with authentication problems. – Facilitation

Answer 2

Beta versions of the git-client and git-plug-in modules have been released now to solve this issue. To quote the JIRA issue.

The git client plugin 2.0.0-beta1 has been released to the experimental update center. It includes git submodule authentication, JGit 4.3, and requires JDK 7. It requires at least Jenkins 1.625 (the first version to mandate JDK 7)

Using the above there is an option under the Additional Behaviours section called:

Use credentials from default remote of parent repository

This solved my issue when using Jenkins 2.11 and the beta versions of the plug-in running a Windows server and slave. I have not checked other build machines. Also you must use the same authentication method, if using http you must use that for submodules as well, if use SSH you must use this for the submodules, trying to mix methods will not work correctly.

-- UPDATE --
Beta versions are not required anymore, please see the following pages:
Git Client Plugin
Git Plugin

Marxmarxian

Answer 3

I just hit this issue. For anyone coming at this with a recent version of Jenkins (post December 2016) try enabling this option in the Advanced sub-modules behaviours section of your git repo config. Use credentials from default remote of parent repository

Hatchet

Answer 4

One solution would be to declare in the global git config file a netrc credential help, which would provide the necessary credentials for any http query coming from git.

git config --global credential.helper "netrc -f C:/path/to/_netrc.gpg -v"

(make sure to use the same account as the one used for running Jenkins)

I use an encrypted netrc file, but you can start some test with an un-encrypted one.

Actuary

I've thought of that as well, but when Jenkins starts to build, it runs this command: using .gitcredentials to set credentials > git config --local credential.helper store --file=/Users/Shared/Jenkins/tmp/git6361286438710002768.credentials # timeout=10 So I think that that solution would get overwritten by Jenkins. – Tubman

@Tubman any chance for your build script to start and do a git config --local of its own? – Actuary

Only if I do it outside of Jenkins, which would defeat the purpose of a build server. I've also tried linking the credential.helper to osxkeychain and such to no avail. – Tubman

Why outside of Jenkins? Your build script is executed by Jenkins. – Actuary

For some reason I didn't think of actually just calling an external shell-script from Jenkins instead of placing all the commands in the shell-block. Let me give that a try. – Tubman

Result is the same as if I would run the commands from within the shell-block in Jenkins so it doesn't make much of a difference. – Tubman

Answer 5

Considering I have tried pretty much all available options to get this working (SSH, .netrc, hardcoded credentials,...) the only option was the one which was mentioned on the bottom of the JENKINS-20941 issue by 'andreg':

Yeah, this issue is a real pain for us too. The only way we could get it working for our Stash/Jenkins setup was to create a read-only user and to hard code this user's credentials in the reference to the submodule. Although bad practise, all users working on the git repo already have at least read-only access, so we didn't feel it was too much of a security concern. e.g. in the parent repo .gitmodules file:

[submodule "shared-library"] path = shared-library url = https://username:[email protected]/scm/project/shared-library.git

and then the Jenkins job has "Recursively update submodules" selected.

Tubman

Answer 6

I actually just moved it to a shell command and in the shell command I told it which credentials helper to use, since I'm on windows it was wincred:

git config --global credential.helper wincred
git submodule init 
git submodule sync 
git submodule update --init --recursive

Lassitude

Answer 7

You can also explicitly provide the credentials if you know the credential name, e.g.

    stage ('Clone') {
        steps {
            checkout scm
            withCredentials([sshUserPrivateKey(credentialsId: 'bitbucket_ssh', keyFileVariable: 'SSH_KEY')]) {
                sh 'GIT_SSH_COMMAND="ssh -i $SSH_KEY" git submodule update --init'
            }
        }
    }

October

Answer 8

This solution worked for me:

1. Make sure ssh credentials are the same for the parent repo and the submodule repo.
2. Setup credentials for the parent repo in Jenkins. (Source Code Management (choose "Git") > Repositories)

3. Setup your .gitmodule file so that it uses the ssh credentials from the parent repo:

   [submodule "foo/repository"]
      path = foo/repository
      url = ssh://[email protected]/repository
   

4. There is a small caveat to this solution. Anytime that someone clones the parent repo, they will need to sync the url to one that they have access to. The first time a user initializes the submodule, they need to first edit the .gitmodules file and change the url:

   [submodule "foo/repository"]
      path = foo/repository
      url = ssh://[email protected]/repository
   

   Then, in the terminal:

   git submodule sync
   git submodule init repository
   git submodule update --remote repository
   

   And then change the url back to the jenkins build url. Unless you sync again, the submodule will use the url associated with the user.

In September 2016, Jenkins is planning to release a new feature that will allow submodules to share the credentials of the parent repository. Then a HTTP url can be used in .gitmodule instead of ssh.

Spinous

hmm, this is a still an issue for a pretty mainline feature it seems – Bairam

Answer 9

I managed to get this working by simply adding a .netrc file with the credentials on linux. Not the most secure solution but if you need to get it working quickly it will get you going.

Autoicous