Goland modules with private gitlab (ssh)

Question

Good day! I try to migrate from glide to go modules (private gitlab repos) and checkout code via ssh

I have a simple project with an import from private gitlab repo.

go.mod looks like:

module my.private.package/modtest

go 1.12

require my.private.package/statistics v1.0.0

when I try to build my app or run test I get:

go: my.private.package/[email protected]: unrecognized import path "my.private.package/statistics" (parse https://my.private.package/statistics?go-get=1: no go-import meta tags ())
go: error loading module requirements

I Tried to add settings to git config:

[url "ssh://[email protected]:9999"]
        insteadOf = https://my.private.package

But still getting this error.

Is there any way to make it work? Thank you.

Daron

Is your code hosting on your custom site? If so, you should have your site returns a meta tag for go get for your package containing repo information. See more at: golang.org/cmd/go/#hdr-Remote_import_paths – Earpiece

leaf bebop, I have a private gitlab server. HTTPS - connection is forbidden - only ssh – Daron

Yes, I am aware of that. What I ask you is to implement Go's Remote Import Protocol, which is specified by the link. The problem is not you can not get the code with git, but rather the step before it: knowing where to get the code from, which is governed by go get, by using Remote Import Protocol. – Earpiece

Answer 1

I've dealt with Go modules and a private GitLab before. Our private GitLab has groups and subgroups. The piece you are likely missing is ~/.netrc and you may have an improper global git configuration.

I've made a GitHub gist for this. You can find it here: https://gist.github.com/MicahParks/1ba2b19c39d1e5fccc3e892837b10e21

You can find the gist pasted below:

Problem

The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.

This assumes your private GitLab is hosted at privategitlab.company.com.

Environment variables

The following environment variables are recommended:

export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com

The above lines might fit best in your shell startup, like a ~/.bashrc.

Explanation

GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using Golang modules on a private GitLab.

GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames listed (like the public module proxy).

Get a personal access token from your private GitLab

To future proof these instructions, please follow this guide from the GitLab docs. I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as well, but have not confirmed this.

Set up the ~/.netrc

In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.

To create the file if it does not exist, run the following commands:

touch ~/.netrc
chmod 600 ~/.netrc

Now edit the contents of the file to match the following:

machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE

Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the previous section.

Common mistakes

Do not set up a global git configuration with something along the lines of this:

git config --global url."[email protected]:".insteadOf "https://privategitlab.company.com"

I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause conflicts with the ~/.netrc.

Bonus: SSH config file

For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up. In order to do this, run the following commands:

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config

Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on most Linux systems.

Then, edit the ~/.ssh/config file to match the following:

Host privategitlab.company.com
  Hostname privategitlab.company.com
  User USERNAME_HERE
  IdentityFile ~/.ssh/id_rsa

Please note the spacing in the above file matters and will invalidate the file if it is incorrect.

Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system. You've already uploaded its public key to GitLab. Here are some instructions.

Chessa

Answer 2

What version of Go are you using? If it's Go 1.13 or later, the default is to download modules through proxy.golang.org. You can change that for a particular set of packages using the GOPRIVATE environment variable.

Here is a quote from go help module-private, which I highly recommend reading in full. Once a module is fetched directly, it should use the same git/ssh logic as before modules.

The GOPRIVATE environment variable controls which modules the go command considers to be private (not available publicly) and should therefore not use the proxy or checksum database. The variable is a comma-separated list of glob patterns (in the syntax of Go's path.Match) of module path prefixes. For example,

GOPRIVATE=*.corp.example.com,rsc.io/private

causes the go command to treat as private any module with a path prefix matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private, and rsc.io/private/quux.

Shackleford

go help module-private appears to be go help private now – Kief

Answer 3

My setup is Gitlab via SSH on a high port example ssh://[email protected]:12345 with the module in the example path /myprojects/modules/go/special-module

1. ~/.ssh/config

Host myhost.com
  Port 12345
  User git

2. The variable GOPRIVATE=myhost.com
3. My module go.mod looks like this

module myhost.com/myprojects/modules/go/special-module.git

4. My project import looks like this (note that without .git it doesn't work)

import (
    special "myhost.com/myprojects/modules/go/special-module.git"
)

Using this set up I don't need any special setting in .gitconfig

Insistence