Go module is found and replaced, But not required

Question

I am getting a weird error when I am trying to build my go code.

$ make install
go version go1.16 windows/amd64
bin/check_go_version 1.14.4
plugin/loader/preload.sh > plugin/loader/preload.go
go fmt plugin/loader/preload.go >/dev/null
go install "-asmflags=all='-trimpath=C:\Users\Deepak Dash\go\src'" "-gcflags=all='-trimpath=C:\Users\Deepak Dash\go\src'" -ldflags="-X "github.com/ipfs/go-ipfs".CurrentCommit=8f9a2b7-dirty" ./cmd/ipfs
cmd\ipfs\daemon.go:32:2: module github.com/ipfs/go-saas-endpoint provides package github.com/ipfs/go-saas-endpoint and is replaced but not required; to add it:
        go get github.com/ipfs/go-saas-endpoint
make: *** [cmd/ipfs/Rules.mk:37: cmd/ipfs-install] Error 1

I have worked previously with go.mods. I have replaced the GitHub package with the local module. and it's detecting the local package.

Thanks, Deepak Dash

Jerz

Answer 1

This is a Go 1.16 issue which is currently investigated in golang/go issue 44529

It includes Jay Conrod's comment:

go mod tidy and go get may both hit the network to look up imported packages that aren't provided by any required module.
If a module is replace locally, the go command will look there first, but I think it may still go out to the network for other prefixes of the module path.

Instead, you can add a requirement on a non-existent version while replacing that version:

go mod edit -require example.com/[email protected] -replace example.com/[email protected]=../local

Adding a replacement, even one without a version on the left side, doesn't automatically add that module to the build list.
If it did, the go command would read its go.mod file and apply its requirements. That could influence selected versions of other modules, even if the replaced module didn't provide any packages.

Bryan C. Mills from Google adds:

go mod tidy should never do a network lookup if it could add a replaced module instead. (see import.go#queryImp())

go get, on the other hand, will perform a network lookup in order to identify the true latest version, taking your replacements into account (query.go#Versions()), and then that version will be replaced instead of downloaded.
It does that so that the latest version added by go get is always consistent with go list -m [⋯]@latest, and so that (if possible) your require directive always specifies a valid version for downstream consumers (if any), so that they won't break when they require your module. (Downstream consumers will not pick up your replace directives, so they need a valid version.)

If you are not using a proxy for the repo in question, that lookup may involve cloning the upstream repo. So that can be a pretty expensive operation. (Note that the official distributions of the go command use proxy.golang.org by default, but the Fedora fork of the go command does not.)

If that network lookup fails, then go get will also fall back to replacement version (query.go#Latest())

Incongruous

Thanks!!! Makes sense, yesterday I updated the go to 1.16 version and it started breaking. – Jerz

To summarize: this is a side-effect of the change to default to -mod=readonly as of Go 1.16. Your build succeeded before because the default behavior was -mod=mod, which causes go build to implicitly add new requirements as needed. With the new default, you need to add those requirements explicitly (such as with go mod tidy or go get) before building. – Sapsago

@Sapsago Good summary. I will monitor (github.com/golang/go/issues/44529, where you left your comment) to see if there will be any resolution or evolution to make that transition (ie " the change to default to -mod=readonly as of Go 1.16") smoother for the users. – Incongruous

go get mycompany.com/path/to/module fixed it for me. Others explained the why of it better, but the key is go get understands that go.mod might have a replace path that's relative and will keep stuff local when it is in fact local. – Naughty

go mod tidy did it for me. – Treponema