How do I enable cloning over SSH for a Gitlab runner?

Question

I am having some trouble cloning large repositories over HTTP on my Windows Gitlab runner. I've tried several methods to do shallow clones or disable clone compression. Still no luck.

Cloning the same repository over SSH works great as a temporary solution and I would like to get this working on our Gitlab CI process.

The issue now stands where I have no idea how to use SSH as a clone method for the gitlab-multi-runner. It just seems to use HTTP as a default, and my only options regarding cloning is whether it will do a full clone or a fetch.

Can someone explain how I could get that clone/fetch to work on a runner over SSH instead of HTTP?

Gitlab Version: GitLab Community Edition 8.10.7

Dolora

There does appear to be an issue opened for this feature (233), but it appears to have been closed without implementation... – Altruism

Answer 1

As a newcomer to gitlab, I've managed to hack a workaround to this issue as I also haven't found a built-in way to change the default cloning process (although here is a recent comment about how it can be done).

By disabling the automatic cloning process, you can effectively override its behavior completely by simply writing your own cloning process in a before_script. Only for the purposes of example does the below show how to accomplish this for HTTP cloning but could be adapted for ssh cloning (if you're trying to use HTTP cloning you should use the built-in cloning process and the config.toml):

1. Create a new user called "gitlab-runner" and generate their user auth token for later use (or in your case, you would generate ssh keys).

2. Disable cloning process for runner by adding the following variable in either your project or group settings: .../settings/ci_cd

   key: GIT_STRATEGY

   value: none

3. Clone your repo in a before_script such as:

before_script:
  ## clean the working directory
  - BUILD_DIR=/home/gitlab-runner/builds/$RUNNER_TOKEN/0
  - CLONE_DIR="$BUILD_DIR/$CI_PROJECT_PATH"
  - cd $BUILD_DIR
  - rm -rf $CLONE_DIR
  - mkdir -p $CLONE_DIR

  ## clone the project each time (inefficient, consider performing fetch instead if it already exists)
  - git clone http://gitlab-runner:$GITLABRUNNER_USER_AUTH_TOKEN@server:8888/${CI_PROJECT_PATH}.git $CLONE_DIR
  - cd $CLONE_DIR

Note: Here are the relevant variables I also configured in step 2 rather than hard coding them in the script:

• RUNNER_TOKEN: "Runner Token" value listed in the Admin "Runners" menu for the particular runner you are trying to run.
• GITLABRUNNER_USER_AUTH_TOKEN: This is the auth token you generated in step 1.

Further Reading:

You can avoid the fake account approach taken above by instead issuing Deploy Keys. Or if security implications of access to any project is a concern, Deploy Tokens are an alternative with more security control. For comparison, see the docs:

Deploy keys are shareable between projects that are not related or don’t even belong to the same group. Deploy tokens belong to either a project or a group.

A deploy key is an SSH key you need to generate yourself on your machine. A deploy token is generated by your GitLab instance, and is provided to users only once (at creation time).

A deploy key is valid as long as it’s registered and enabled. Deploy tokens can be time-sensitive, as you can control their validity by setting an expiration date to them.

You can’t log in to a registry with deploy keys, or perform read / write operations on it, but this is possible with deploy tokens. You need an SSH key pair to use deploy keys, but not deploy tokens.

Burkholder

Answer 2

According to:

https://docs.gitlab.com/ee/ci/ssh_keys/README.html

You need to:

1. Create a new SSH key pair with ssh-keygen
2. Add the private key as a Secret Variable to the project
3. Run the ssh-agent during job to load the private key.

Example gitlab_ci.yml:

before_script:
  # Install ssh-agent if not already installed, it is required by Docker.
  # (change apt-get to yum if you use a CentOS-based image)
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'

  # Run ssh-agent (inside the build environment)
  - eval $(ssh-agent -s)

  # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  - ssh-add <(echo "$SSH_PRIVATE_KEY")

  # For Docker builds disable host key checking. Be aware that by adding that
  # you are suspectible to man-in-the-middle attacks.
  # WARNING: Use this only with the Docker executor, if you use it with shell
  # you will overwrite your user's SSH config.
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  # In order to properly check the server's host key, assuming you created the
  # SSH_SERVER_HOSTKEYS variable previously, uncomment the following two lines
  # instead.
  # - mkdir -p ~/.ssh
  # - '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'

Congeries

This doesn't solve the problem of gitlab-runner still trying to clone the repo over https, even with keys configured. See @Burkholder answer for a way to do that. – Furry

Answer 3

I had a similar problem that necessitated the use of cloning via ssh: using the virtualbox executor with very old guest linux OSes. I was able to get around it by doing a few small configuration changes:

1. Create a deploy key for access to the project.

2. Force the user account that will perform the clone to use the deploy key. In my virtualbox case, I modified the ssh configuration for the user that's configured for virtualbox in /etc/gitlab-runnner/config.toml.

~/.ssh/config

Host gitlab.example.com
  Preferredauthentications publickey
  IdentityFile ~/.ssh/deploy-key

3. Configure the runner to perform the clone via ssh in /etc/config.toml.

/etc/config.toml

[[runners]]

  # [...]

  environment = ["GIT_STRATEGY=none"]
  pre_build_script = '''
    # Fetching using ssh (via pre_build_script in config.toml)
    if [ -d "${CI_PROJECT_DIR}" ]; then rm -rf "${CI_PROJECT_DIR}"; fi
    mkdir -p "${CI_PROJECT_DIR}"
    cd "${CI_PROJECT_DIR}"
    git init
    git remote add origin "ssh://git@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
    git fetch origin "${CI_COMMIT_SHA}"
    git reset --hard FETCH_HEAD
  '''

  # [...]

Here's a breakdown of the additions to config.toml:

• The GIT_STRATEGY=none environment variable disables the runner's internal git cloning mechanism. (See the Git Strategy section of the CI/CD reference)
• The pre_build_script performs the actual clone using predefined CI/CD environment variables. In my case, this is a bash script to perform something similar to what a GIT_STRATEGY=fetch might do.
• If pre_build_script is multi-line, the output of the runner will only show the first line. Having a comment as the first line helps add clarity to the runner output.
• pre_clone_script is not used here. It's disabled since the environment has GIT_STRATEGY=none set.

Corporeal

That's pretty cool, I was actually looking for this solution a couple of months ago, before you actually wrote it :) – Shingles

Thanks, @aledpardo! Hopefully it's useful for some folks out there. :) – Corporeal

Answer 4

This worked for me, Add this in your .gitlab-ci.yml

script:
  - git remote set-url --push origin git@$CI_SERVER_HOST:$CI_PROJECT_PATH.git

This will set your push url in .git/config file and runner will use this while pushing code back to your git repo instead of the url that uses ci-token.

Prove

Gitlab server: Ubuntu 18.04, gitlab-ci 15.0. Gitlab-runner server: Ubuntu 14.04, gitlab-runner 11.11.4 Solution was in /etc/gitlab-runner/config.toml, section [[runners]] pre_clone_script = '''cd "${CI_PROJECT_DIR}" && git config url."ssh://git@$CI_SERVER_HOST:".insteadOf https://gitlab-ci-token:$CI_JOB_TOKEN@$CI_SERVER_HOST''' – Starveling

Answer 5

I was having issues cloning over https with a very old version of git. I ended up making a template that cloned the repo if needed and otherwise fetched the latest changes and cleaned up the repo to the bare clean state.

You need deploy keys set up for all the repos involved, including submodules on the same gitlab server. Set up an SSH key as per the Using SSH keys documentation, and add it as a Deploy Key for all repos that need to be cloned.

This is the template that can be extended whenever needed:

.ssh_cloning:
    variables:
        GIT_STRATEGY: none
    before_script:
        # set up ssh keys for cloning
        - eval $(ssh-agent -s)
        - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
        # clean the working directory
        - |
            if [ ! -d "${CI_PROJECT_DIR}/.git" ]
            then
                mkdir -p "${CI_PROJECT_DIR}"
                cd "${CI_PROJECT_DIR}"
                git init
                git remote add origin "ssh://git@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
            fi
        - cd "${CI_PROJECT_DIR}"
        # clone the project
        - git fetch origin
        - git checkout "${CI_COMMIT_SHA}"
        - git reset --hard HEAD
        - git submodule update --init --recursive
        # clean up files from the last build
        - git clean -ffdx

You can include it in other jobs with something like this (untested):

include:
    - 'ssh_cloning.yml'

stages:
    - build

do_git_stuff:
    stage: build
    extends: .ssh_cloning
    script:
        - git status

Undis

Answer 6

All solution mentioned here do work in 99% of all cases. Either the repository is not clean (you can't run git clean -ffdx as this would also remove the pipeline artifacts which were already downloaded) and/or the cloning process is not efficient.

The only working solution is to add the SSH cloning feature to the Gitlab Runner and do not configure anything. Unfortunately you need to set the clone_url configuraton option in the config.toml. Thus you need a self hosted runner.

Check the fork of Gitlab Runner at Hapag-Lloyd/GitLab-Runner (I am the author of this project). Feel free to add a MR request for other versions as 14.8 of the Runner is supported at the moment.

Sorenson