go mod tidy error message: "but go 1.16 would select"

Question

When I run go mod tidy a few packages show an error

> go mod tidy

github.com/myrepo/myproj imports
    go.k6.io/k6 imports
    go.k6.io/k6/cmd imports
    github.com/fatih/color loaded from github.com/fatih/[email protected],
    but go 1.16 would select v1.13.0

To upgrade to the versions selected by go 1.16:
    go mod tidy -go=1.16 && go mod tidy -go=1.17
If reproducibility with go 1.16 is not needed:
    go mod tidy -compat=1.17
For other options, see:
    https://golang.org/doc/modules/pruning

I have go 1.17.9 installed. What's the meaning of the error and why is it being triggered?

Merry

Answer 1

This error is related to module graph pruning introduced in Go 1.17.

With Go 1.16, the module graph for Minimal Version Selection used to include the full module graph, whereas with 1.17 the graph includes only up to transitive dependencies (with some exceptions, see the link above).

Now to understand what triggers the error, you might want to look at the Go 1.17 release notes:

By default, go mod tidy verifies that the selected versions of dependencies relevant to the main module are the same versions that would be used by the prior Go release (Go 1.16 for a module that specifies go 1.17) [...]

So when you run go mod tidy, it reports that Go 1.16 "would select" a version of a transitive dependency (github.com/fatih/color) that is different from the one that the pruned graph of Go 1.17 would.

This is relevant for build reproducibility, because go.sum contains the checksums for the current Go version specified in go.mod and the previous one. In case of Go 1.17 and Go 1.16 where the module graph effectively can change, go.sum would be inconsistent.

The error message suggests two fixes.

1. go mod tidy -go=1.16 && go mod tidy -go=1.17 — this selects the dependency versions as Go 1.16 and then as Go 1.17

2. go mod tidy -compat=1.17 — this simply removes the Go 1.16 checksums (hence the tip "reproducibility with go 1.16 is not needed").

The error should not present itself anymore after you upgrade to Go 1.18, because then the module graph will be loaded the same as in Go 1.17.

Wheelwright

Answer 2

Simple explanation

The error but go 1.16 would select means there is now a deeper problem with how your compiled software (compiled binary) behaves after being compiled with Go 1.16 (or below) instead of Go 1.17 (or above).

What introduced this problem?: this may be entirely out of your control, for example an innocent change in one of your dependencies can introduce it as a side effect. (As recently seen with golang.org/x/oauth2 and similar which has broken a lot of builds around the world.)

Can I simply avoid running go mod tidy? Yes, but this does nothing about your actual problem.

What is the practical impact then for me? It's that as of now you have no build reproducibility between Go 1.16 and 1.17. If you use Go 1.16 to build or test, the behavior of your program may be subtly different compared to that of Go 1.17+. The compilation of your program takes different versions of the dependencies. Very slightly different, but different, and it's more important than the intimate details of the underlying algorithm.

Force build compatibility with Go 1.17 only (or above)

1. Document/communicate that nobody is supposed to compile your code with Go 1.16 or below.

2. Make sure your continuous integration is not using Go 1.16 or below.

3. In all your scripts, Makefiles, pipelines, etc., change command go mod tidy to:

go mod tidy -compat=1.17

Is this a migration? I have never used Go 1.16!

You might think that declaring a version go 1.17 in your go.mod enforces usage of that Go version or higher. Even some Go 1.16 tools in that case do fail with go.mod file indicates go 1.17, but maximum supported version is 1.16, enforcing that intuition. Makes sense, right? Nope.

The brutish reality is that some codebases of that kind (and maybe yours too) can be compiled with Go 1.16 or Go 1.15 as long as the compiled module doesn't include features only added in later Go versions! The core team doesn't want to silently introduce problems for such contrived build processes. This is why you are faced with the decision to explicitly keep or explicitly drop that kind of backward compatibility.

Alternative: use the Go 1.16 dependency algorithm

go mod tidy -go=1.16 && go mod tidy -go=1.17

bumping the last number to your max version, so if you are on 1.18:

go mod tidy -go=1.16 && go mod tidy -go=1.18

The latter -go=1.17 (or -go=1.18) flag declares that you want to cap the language features at 1.17 (or 1.18).

The former -go=1.16 is only transient and immediately overwritten. Why is it needed then? Well, it is needed for a side effect: it leaves a mark in go.mod. The 1.17 (or 1.18) sees the mark and because of that it doesn't use the novel algorithm of dependency pruning. Instead it chooses the same versions as 1.16 would and persists them into the go.mod.

Milliner