How to use Go with a private GitLab repo

Asked 17/4, 2015 at 19:19 Answered 7/6 at 11:47

GitLab is a free, open-source way to host private .git repositories but it does not seem to work with Go. When you create a project it generates a URL of the form:

[email protected]:private-developers/project.git

where:

1.2.3.4 is the IP address of the gitlab server
private-developers is a user group which has access to the private repo

Golang 1.2.1 doesn't seem to understand this syntax.

go get [email protected]:private-developers/project.git

results in:

package [email protected]/project.git: unrecognized import path "[email protected]/project.git"

Is there a way to get this to work?

Laurencelaurene answered 17/4, 2015 at 19:19 Comment(5)

that's not how go get works, it uses import paths, not direct scm uris. Does your gitlab have https git access (like github)? – Ganoid 17/4, 2015 at 19:25

I'm not sure. It was installed via this procedure: cloud.google.com/solutions/gitlab . Is there a way i can tell? – Laurencelaurene 17/4, 2015 at 19:49

I'm not really familiar with gitlab, but I know they added support for go get on public projects: github.com/gitlabhq/gitlabhq/pull/5958. Worst case you can always just do the initial clone of the repo into your GOPATH manually – Ganoid 17/4, 2015 at 19:56

regrettably i have to use a private repo for this. I am using the 'worst case' for now :). – Laurencelaurene 17/4, 2015 at 19:59

Possible duplicate of What's the proper way to "go get" a private repository? – Papain 23/10, 2017 at 21:46

This issue is now resolved in Gitlab 8.* but is still unintuitive. The most difficult challenge indeed is go get and the following steps will allow you to overcome those:

Create an SSH key pair. Be sure to not overwrite an existing pair that is by default saved in ~/.ssh/.
```
ssh-keygen -t rsa -b 4096
```
Create a new Secret Variable in your Gitlab project. Use SSH_PRIVATE_KEY as Key and the content of your private key as Value.

Modify your .gitlab-ci.yml with a before_script.

before_script:
  # install ssh-agent if not already installed
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  # run ssh-agent
  - eval $(ssh-agent -s)
  # add the SSH key stored in SSH_PRIVATE_KEY
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  # for Docker builds disable host key checking
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

Add the public key from the key pair created in step 1 as a Deploy Key in the project that you need to go get.

Paladin answered 17/5, 2017 at 23:23 Comment(4)

regrettably i can't verify that this works as I'm not currently using gitlab. If someone can verify this i'll mark it as the answer. – Laurencelaurene 18/5, 2017 at 0:54

Thanks @JamesFremen, I don't know that anyone will jump on it but I've had several people ask me about it and it seems to work well. – Paladin 14/6, 2017 at 20:24

This worked for me after I added one more line to the before_script section: - git config --global url."[email protected]:".insteadOf "https://gitlab.com/" – Solidstate 19/7, 2017 at 16:37

Disabling StrictHostKeyChecking is not recommended as it constitutes a security risk. So this is not really a solution. – Sthilaire 17/6 at 8:19

Run this command:

git config --global url."[email protected]:".insteadOf "https://1.2.3.4/"

Assuming you have the correct privileges to git clone the repository, this will make go get work for all repos on server 1.2.3.4.

I tested this with go version 1.6.2, 1.8, and 1.9.1.

Papain answered 15/6, 2016 at 19:34 Comment(1)

simply does not work – Cal 3/1, 2023 at 20:21

Easiest way with Gitlab

before_script:
  - git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/".insteadOf https://gitlab.com/
  - go env -w GOPRIVATE=gitlab.com/${CI_PROJECT_NAMESPACE}

See more details here: https://docs.gitlab.com/ee/user/project/new_ci_build_permissions_model.html#dependent-repositories

Handbook answered 23/3, 2020 at 7:38 Comment(2)

This does indeed work, and it is very clean. The only issue here is that this documentation link is broken 🔗. I was able to find these: docs.gitlab.com/ee/ci/variables/predefined_variables.html docs.gitlab.com/ee/ci/jobs/ci_job_token.html They define the available environment vars, and the one for CI_JOB_TOKEN has an example for a clone command that uses the same url with the username:token syntax as you see here. This example followed the answer in https://mcmap.net/q/22507/-how-to-convert-git-urls-to-http-urls to create the git config command. – Merrie 19/11, 2022 at 20:20

This must be marked as a correct answer. – Organology 15/7, 2023 at 7:44

This issue is now resolved in Gitlab 8.* but is still unintuitive. The most difficult challenge indeed is go get and the following steps will allow you to overcome those:

Create an SSH key pair. Be sure to not overwrite an existing pair that is by default saved in ~/.ssh/.
```
ssh-keygen -t rsa -b 4096
```
Create a new Secret Variable in your Gitlab project. Use SSH_PRIVATE_KEY as Key and the content of your private key as Value.

Modify your .gitlab-ci.yml with a before_script.

before_script:
  # install ssh-agent if not already installed
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  # run ssh-agent
  - eval $(ssh-agent -s)
  # add the SSH key stored in SSH_PRIVATE_KEY
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  # for Docker builds disable host key checking
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

Add the public key from the key pair created in step 1 as a Deploy Key in the project that you need to go get.

Paladin answered 17/5, 2017 at 23:23 Comment(4)

regrettably i can't verify that this works as I'm not currently using gitlab. If someone can verify this i'll mark it as the answer. – Laurencelaurene 18/5, 2017 at 0:54

Thanks @JamesFremen, I don't know that anyone will jump on it but I've had several people ask me about it and it seems to work well. – Paladin 14/6, 2017 at 20:24

This worked for me after I added one more line to the before_script section: - git config --global url."[email protected]:".insteadOf "https://gitlab.com/" – Solidstate 19/7, 2017 at 16:37

Disabling StrictHostKeyChecking is not recommended as it constitutes a security risk. So this is not really a solution. – Sthilaire 17/6 at 8:19

GitLab version 11.8+ and Go version 1.13+ will work with BASIC auth by using your GitLab personal token. Go to Settings -> Access Tokens in your Gitlab, add a personal access token or use your existing one. In your ~/.netrc file, add following lines:

machine <your GitLab domain> (e.g. gitlab.com)
login <your GitLab id>
password <your GitLab personal access token>

Then you should be able to do go get locally.

If you need to build it in CI, then add following line in your .gitlab-ci.yml file:

before_script:
    - echo -e "machine <your GitLab domain>\nlogin gitlab-ci-token\npassword ${CI_JOB_TOKEN}" > ~/.netrc

Refresh answered 16/4, 2020 at 18:48 Comment(0)

If go get can't fetch the repo, you can always do the initial clone with git directly:

git clone git@gitlab:private-developers/project.git $GOPATH/src/gitlab/private-developers/project

The tools will then work normally, expect for go get -u which will require the -f flag because the git remote doesn't match the canonical import path.

Ganoid answered 17/4, 2015 at 20:2 Comment(1)

yes, i think this is the workaround i'm currently using. – Laurencelaurene 18/4, 2015 at 21:13

Gitlab does support go get natively.

go get will issue an http request to the url you provide and look for meta tags that point to the exact source control path.

For my gitlab installation this is mygitlabdomain.com/myProject/myRepo. For you I assume this would be 1.2.3.4/private-developers/project.

Unfortunately it only appears to give the http scm path, not the ssh path, so I had to enter my credentials to clone. You can easily fiddle with the remote in your local repository after it clones if you want to update to the ssh url.

You can test the url by poking http://1.2.3.4:private-developers/project?go-get=1 and viewing source and looking for the meta tag.

Dressingdown answered 18/4, 2015 at 3:25 Comment(2)

I've posted an answer as well.. it seems like a fix is in gitlab 7.9. – Laurencelaurene 18/4, 2015 at 22:39

thanks for all the help.. hopefully this is working smoothly in the near future – Laurencelaurene 18/4, 2015 at 22:48

From dep version 5.2, dep supports private repositories for Gitlab private repositories.

On .netrc file, you can provide your Gitlab username and access token for accessing private repositories.

Create .netrc file in your $HOME directory

$ touch $HOME/.netrc

Edit your .netrc with your Gitlab credentials

machine gitlab.<private>.com
login <gitlab-username>
password <gitlab-access-token>

... (more private repositories if needed)

In your Go repository, run the dep command to resolve private packages. In this case,

$ dep ensure -v

Sulcate answered 9/5, 2019 at 4:18 Comment(1)

Creating the .netrc file resolved my issue in my ubuntu – Flashcube 19/9, 2021 at 12:33

The way I usually do it is:

Ensure you are using SSH.

once that's done you can configure your git to use ssh instead https

If you are using Mac OX. you can run vim ~/.gitconfig and add

[url "[email protected]:"]
insteadOf = https://gitlab.com/

once configured you can run

GOPRIVATE="gitlab.com/your_username_or_group" go get gitlab.com/name_or_group/repo_name

I hope that helps.

Ajani answered 13/12, 2019 at 6:53 Comment(0)

For HTTPS private gitlab repo, @Rick Smith's answer is enough. Here's a compensation for HTTP repo, first run the command:

git config --global url."[email protected]:".insteadOf "http://mygitlab.com/"

then use below go get command to get the golang project:

go get -v  -insecure  mygitlab.com/user/repo

Ashliashlie answered 15/4, 2019 at 9:42 Comment(1)

At this time, go: -insecure flag is no longer supported; use GOINSECURE instead – Intinction 20/2, 2023 at 14:20

For the record, this works outside of go using gitlab 7.3.2 and, as JimB has observed, can be used as a workaround. I find that i do get prompted for username/password, even though an SSH key is registered with gitlab:

git clone http://1.2.3.4/private-developers/project.git

Alternatively i can use the SSH equivalent which, since i have an SSH key registered with gitlab, avoids the prompts:

git clone [email protected]:private-developers/project.git

Neither works with go currently. A fix may be in 7.9 but i haven't had a chance to test it: upcoming bugfix

Laurencelaurene answered 18/4, 2015 at 22:36 Comment(2)

Just a heads up. Although the bug was closed, the issue still remains. – Laurencelaurene 22/4, 2015 at 16:52

@JamesFreman I'm on gitlab 8.2, and still getting http urls. Any idea what is going on? – Dressingdown 15/12, 2015 at 20:17

You can setup your git credentials and Go will use them:

generate a unique password on your github (somewhere in settings).
git config credential.helper store
echo https://your-github-username:[email protected] >> ~/.git-credentials
profit.

Monte answered 18/4, 2015 at 0:58 Comment(1)

He is talking about gitlab, not github. Gitlab is an open source project for private repository hosting. – Dressingdown 18/4, 2015 at 3:26

Found an article that covers this in a decent amount of detail here, including how to get this to work in a Docker container. Configuring Gitlab repos to be cloned with SSH rather than HTTPS and using GOPRIVATE seems to be the main two things to keep in mind.

Slipstream answered 7/6 at 11:47 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags