Jenkins - Unable to git clone from slave node. SSH keys

Question

I have just set up my first Jenkins slave. I run the build, and I hit problems with the SSH keys. The master Jenkins server is running under the user 'jenkins'. I have set up the SSH keys such that I can SSH from the master to the slave without a password.

e.g. From the master:

jenkins@master:~$ ssh slave
Last login: Tue Apr 17 10:30:22 2012 from masterjenkins.com
$ whoami
jenkins

So that proves that the slave node is also running under the 'jenkins' user. (I have copied the public ssh key from the jenkins@slave to the remote git server). And I am able to issue the git clone manually from the slave, but when I kick off the build from the master I get these kinds of messages:

    ERROR: Error cloning remote repo 'origin' : Could not clone git@host:abc
hudson.plugins.git.GitException: Could not clone git@host:abc
Caused by: hudson.plugins.git.GitException: Error performing command: git clone --progress -o origin git@host:abc /var/lib/jenkins/workspace/abc_build
Command "git clone --progress -o origin git@host:abc /var/lib/jenkins/workspace/abc_build" returned status code 128: Initialized empty Git repository in /var/lib/jenkins/workspace/abc_build/.git/
Host key verification failed.
fatal: The remote end hung up unexpectedly
Caused by: hudson.plugins.git.GitException: Command "git clone --progress -o origin git@host:abc /var/lib/jenkins/workspace/abc_build" returned status code 128: Initialized empty Git repository in /var/lib/jenkins/workspace/abc_build/.git/
Host key verification failed.
fatal: The remote end hung up unexpectedly
Trying next repository
ERROR: Could not clone repository
FATAL: Could not clone

So it is still hinting that my SSH keys are not set up correctly. Can anyone tell me what keys I need to copy where?

Many thanks, ns

Essequibo

also host key verification failed seems to indicate your Jenkins user has never never ssh'd into that server and you haven't accepted the host key, try sshing from the CLI as the jenkins user to make sure it works, and accept the host key. – Livre

@Livre This could end up being the issue. I had originally thought you was trying to ssh from the same machine that Jenkins is trying to clone from. – Daugavpils

Got the bastard! Thanks for all your input. I copied and pasted the command jenkins was attempting to run on the slave: git clone --progress -o origin git@host:abc /var/lib/jenkins/workspace/abc and it turned out I had some offending keys in /root/.ssh/known_hosts After removing these and connecting to the git repo once more, it worked! – Essequibo

Answer 1

Based on the clone URL it appears you are mixing up two different authentication methods. You are trying to SSH into host as the user git not jenkins. Typically when you host your own GIT repositories and clone with git@servername:reponame you use something like gitolite.

Did you set anything up like gitolite?

Try ssh'ing like this instead as the jenkins user.

ssh git@slave 

Then see what that returns. This is an SSH more in line with the git@host:abc you are doing.

If you didnt setup anything else on your server machine, then change your clone url to jenkins@host:pathtorepo

Update

/home/git/.ssh/authorized_keys

Should have an entry like this: (This is all on ONE line)

# gitolite start
command="/home/git/bin/gl-auth-command jenkins",no
-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAt3+od84Gc9NBVrVb3MKjekHcBDwXXONnVYMNVpuRadoz/FPJTkOIxozKVPJDPI670O252giYpF59sOKqAJL0xEVUrhq8cDFuFwQsSAp0ed1kp/GRxx+pwytL58rcVJEHAy2DkD1z5HlNaZyvIxQyfLTnYfuL1Hx6Qe7dal7mXO0= keycomment
# gitolite end

Add repository permissions to jenkins in gitolite: (You may have to clone on the same machine that is hosting your repositories, as the gitolite user)

git clone git@host:gitolite-admin 
cd gitolite-admin
cd conf
vi gitolite.conf

Now find the entry for 'abc' or add one if it doesn't exist

repo    abc
  RW+            = jenkins

Now commit and push changes

git commit -a -m "Adding user jenkins to repo abc"
git push

Now do your ssh git@host again to see if gitolite tells you about your new permissions.

Daugavpils

Hi, sorry yes I forgot to add that we use gitolite to manage our git users. ssh'ing from the master to slave as ssh git@slave just prompts for a password. One thing that is bugging me is that I couldn't easily find an easy way to see who jenkins runs as on the slave. Once logged onto the slave, I did a ssh-keygen and saved the files in the default location which was /home/jenkins/.ssh/id_rsa. I then copied the public key to gitolite. – Essequibo

@Essequibo That means gitolite or something else is not setup correctly. Are you sure the authorized_keys is setup correctly on your server? I use gitlab to manage my gitolite instance. Until you can successfully ssh git@host without a password, your clones won't work. First thing to check is that the authorized_keys on the server for the git user and your jenkins name in it with the correct public key. – Daugavpils

From the slave: $ ssh git@host PTY allocation request failed on channel 0 hello jenkins, this is gitolite v2.1-31-gf0cedeb running on git 1.7.0.4 the gitolite config gives you the following access: @R_ @W_ testing R W abc Connection to host closed. – Essequibo

@Essequibo Is the repo name called 'testing' that you are trying to clone from Jenkins? – Daugavpils

Hey, no it is called 'abc'. Is it normal for gitolite to close the connection straight after printing the permissions? Sorry, this is all new tech to me. – Essequibo

@Essequibo Then it appears gitolite is only configured to give you access to the repo named 'testing'. I'll update my answer on how to add the repo 'abc' – Daugavpils

@nonshatter, "One thing that is bugging me is that I couldn't easily find an easy way to see who jenkins runs as on the slave" - one way to do that is to create a job attached to the slave that runs whoami in a shell build step. – Violet

@Essequibo We can bring this to chat if need be. Just let me know. – Daugavpils

Okay, here's an update: I run whoami as a jenkins build step and it tells me that the slave is actually running as root. I have copied the public key from the master to authorized keys on the slave. Now I can ssh without password as root->slave. I chown'd the jenkins FS root dir to be owned by root. Now I am get slightly different messages: Command "git fetch -t git@host:abc +refs/heads/*:refs/remotes/origin/*" returned status code 128: Host key verification failed. fatal: The remote end hung up unexpectedly ... Any thoughts? – Essequibo

@charlesb if you are getting the host key problem you need to run the ssh command and accept the host key or add it to your known_hosts file. – Daugavpils