Authenticate Jenkins CI for Github private repository

Question

I'd like for Jenkins to automagically fetch data from my private repository hosted on Github. But I have no idea how to accomplish that task.. Tried the documentation, generating ssh-key for jenkins user and all what I can see is: "unable to clone the repo". I've checked URLs - they are valid.

Any clues, maybe you know some docs/blogs/whatever which are describing this kind of stuff?

Robyn

I answer this a similar question, you can see the answer in the link below: jenkins & GitHub – Cyano

Answer 1

Perhaps GitHub's support for deploy keys is what you're looking for? To quote that page:

When should I use a deploy key?

Simple, when you have a server that needs pull access to a single private repo. This key is attached directly to the repository instead of to a personal user account.

If that's what you're already trying and it doesn't work, you might want to update your question with more details of the URLs being used, the names and location of the key files, etc.

---

Now for the technical part: How to use your SSH key with Jenkins?

If you have, say, a jenkins unix user, you can store your deploy key in ~/.ssh/id_rsa. When Jenkins tries to clone the repo via ssh, it will try to use that key.

In some setups, you cannot run Jenkins as an own user account, and possibly also cannot use the default ssh key location ~/.ssh/id_rsa. In such cases, you can create a key in a different location, e.g. ~/.ssh/deploy_key, and configure ssh to use that with an entry in ~/.ssh/config:

Host github-deploy-myproject
    HostName       github.com
    User           git
    IdentityFile   ~/.ssh/deploy_key
    IdentitiesOnly yes

Because all you authenticate to all Github repositories using [email protected] and you don't want the above key to be used for all your connections to Github, we created a host alias github-deploy-myproject. Your clone URL now becomes

git clone github-deploy-myproject:myuser/myproject

and that is also what you put as repository URL into Jenkins.

(Note that you must not put ssh:// in front in order for this to work.)

Mcglone

Yeah - already managed to solve my problem - I've added ssh key directly to the repo (as deploy key) and not to the account as before. – Robyn

Bur how do you created a key for jenkins? – Tiepolo

The "deploy key" is just any old SSH key. What I did is run ssh-keygen as the user Jenkins runs as ("jenkins" on my Ubuntu server). I then added ~jenkins/.ssh/id_rsa.pub to the deploy keys section of the repository on github. – Cousins

This answer doesn't help much. Looking for the process of making the keys particularly. I think that you have to login as whatever system account tomcat/jenkins is running as and gen the keys then scoop them up out of /var/empty. – Petrarch

on some installs you would need to out this not into ~ directory. But into /var/lib/jenkins/.ssh/ for the default jenkins user to use those keys! – Jerrodjerrol

To follow up on the comment by @garmoncheg, note that /var/lib/jenkins is the home directory (~) for the jenkins user. – Chobot

Any clue how to use this method when updating submodules for myuser/myproject ?? opened up a question at #25536132 – Dutiable

Does anyone know how to get deploy hooks working with this? I'm seeing errors akin to Could not match github-deploy-myproject:myuser/myproject in the hook log. I've entered that as my Repo URL and builds do work so it can access GitHub. It's just the post from GitHub that fails to trigger the build. – Acclamation

To be clear, this solution does not currently work with Github Post-hooks, so triggering jobs does not work. - issues.jenkins-ci.org/browse/JENKINS-18298 – Trusteeship

@Acclamation I found that triggering builds using the github plugin seemed to require that the Repository URL matched the SSH clone URL, as the plugin seems to reconstruct that URL from the post-hook and trigger builds that have a matching repository URL. I have multiple projects, so ended up creating a dummy user with a single SSH key and access to all the private repos, rather than a deployment key, but I suspect changing Host github-deploy-myproject to Host github.com and using [email protected]:myuser/myproject for repository URL may be what you need (based on my setup, YMMV of course) – Convolute

Answer 2

One thing that got this working for me is to make sure that github.com is in ~jenkins/.ssh/known_hosts.

Macaroon

This fixed the issue I was having where after setting up a key pair, a git push was failing – Convolute

In my case the easiest way to do this is to do 'sudo su jenkins' as it isn't possible to log in as the jenkins user properly. Once you have the jenkins identity, you can do a manual ssh login to github/bitbucket and accept the remote host key on behalf of the jenkins user. – Banquer

But what if you making Jenkins Initialization part of your development environment 'bootstrap'. The 'manual' aspect of this doesn't work – Gigantic

Answer 3

If you need Jenkins to access more then 1 project you will need to:
1. add public key to one github user account
2. add this user as Owner (to access all projects) or as a Collaborator in every project.

Many public keys for one system user will not work because GitHub will find first matched deploy key and will send back error like "ERROR: Permission to user/repo2 denied to user/repo1"

http://help.github.com/ssh-issues/

Tuft

The answer about using a deploy key works great if you just have a single repository. But when you want a CI server to build projects across multiple repos you're immediately in the position of managing several sets of keys (one pair per repo) and it becomes much easier to take the approach listed in this answer. – Plait

This guy's guide explains how to set it via different deploy keys using ~/.ssh/config : gist.github.com/victorborda/2871029 – Ramify

@JorgeOrpinel, I believe the approach in the link can prevent github webhooks from triggering builds using the Github plugin. I found a dummy user with a single key and access to all of the repos worked better when I also wanted builds to be triggered by a webhook, because I needed the Repository URL in the build config to match the github clone URL, see my other comment – Convolute

Answer 4

Jenkins creates a user Jenkins on the system. The ssh key must be generated for the Jenkins user. Here are the steps:

sudo su jenkins -s /bin/bash
cd ~
mkdir .ssh // may already exist
cd .ssh
ssh-keygen

Now you can create a Jenkins credential using the SSH key On Jenkins dashboard Add Credentials

select this option

Private Key: From the Jenkins master ~/.ssh

Planoconvex

Answer 5

I had a similar problem with gitlab. It turns out I had restricted the users that are allowed to login via ssh. This won't affect github users, but in case people end up here for gitlab (and the like) issues, ensure you add git to the AllowUsers setting in /etc/ssh/sshd_config:

# Authentication:
LoginGraceTime 120
PermitRootLogin no
StrictModes yes
AllowUsers batman git

Commemoration

Answer 6

Another option is to use GitHub personal access tokens:

• Go to https://github.com/settings/tokens/new
• Add repo scope
• In Jenkins, add a GitHub source
• Use Repository HTTPS URL
• Add the HTTPS URL of the git repo (not the SSH one, eg. https://github.com/my-username/my-project.git)
• Add credential
  • Kind: Username with Password
  • Username: the GitHub username
  • Password: the personal access token you created on GitHub
  • ID: something like github-token-for-my-username

I tested this on Jenkins ver. 2.222.1 and Jenkins GitHub plugin 1.29.5 with a private GitHub repo.

Nook

Answer 7

An alternative to the answer from sergey_mo is to create multiple ssh keys on the jenkins server.

(Though as the first commenter to sergey_mo's answer said, this may end up being more painful than managing a single key-pair.)

Markettamarkey

Now I see why simply posting a URL is a terrible strategy for answers. The link above is dead. – Deandra