Content of go.sum and modules really used by a go application
Asked Answered
O

2

16

I'm trying to compare the behavior of go mod tidy (and the resulting content of go.sum) to the output of go list -m all. Reading the docs, I understand go.sum contains the whole list of dependent modules declared in go.mod and in dependencies' go.mod files, go list -m all shows the modules really loaded during the execution. As an example, an application including logrus and prometheus like this:

go.mod

module mytest

go 1.14

require (
        github.com/prometheus/common v0.4.0
        github.com/sirupsen/logrus v1.8.1
)

main.go

package main

import "github.com/sirupsen/logrus"
import "github.com/prometheus/common/version"

func main() {
  logrus.Info("Hello World")
  logrus.Infof("Prometheus info: %v", version.Info())
}

After go mod tidy, go.sum shows both logrus v1.8.1, requested by the go.mod, and 1.2.0, dependency of prometheus v0.4.0; go list -m all shows only v1.8.1.

go.sum

[...]
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE=
github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
[...]

output of go list

[...]
github.com/sirupsen/logrus v1.8.1
[...]

Is it correct to say the modules really used by the application are listed by go list -m all?

The underlying problem is that a static code analysis detects insecure module versions listed in go.sum, but actually these versions don't show up in go list -m all, hence they shouldn't be really used by the application, but only downloaded during build phase to select the proper minimal version.

Some reference:

https://go.dev/ref/mod#go-mod-tidy

go mod tidy acts as if all build tags are enabled, so it will consider platform-specific source files and files that require custom build tags, even if those source files wouldn’t normally be built.

https://go.dev/ref/mod#go-sum-files

The go.sum file may contain hashes for multiple versions of a module. The go command may need to load go.mod files from multiple versions of a dependency in order to perform minimal version selection. go.sum may also contain hashes for module versions that aren’t needed anymore (for example, after an upgrade).

https://github.com/golang/go/wiki/Modules#is-gosum-a-lock-file-why-does-gosum-include-information-for-module-versions-i-am-no-longer-using

[...]In addition, your module's go.sum records checksums for all direct and indirect dependencies used in a build (and hence your go.sum will frequently have more modules listed than your go.mod).

https://github.com/golang/go/wiki/Modules#version-selection

The minimal version selection algorithm is used to select the versions of all modules used in a build. For each module in a build, the version selected by minimal version selection is always the semantically highest of the versions explicitly listed by a require directive in the main module or one of its dependencies.

As an example, if your module depends on module A which has a require D v1.0.0, and your module also depends on module B which has a require D v1.1.1, then minimal version selection would choose v1.1.1 of D to include in the build (given it is the highest listed require version). [...] To see a list of the selected module versions (including indirect dependencies), use go list -m all.

Originality answered 26/11, 2021 at 17:24 Comment(2)
This "static code analysis" is making incorrect assumptions about the purpose of go.sum. It exists to guarantee reproducible builds where the go.mod fully lists your module's requirements, by proving that none of your dependencies' releases have had their content changed. It is an optional file, if you trust all your dependencies you can throw it away. The older logrus is listed because one of your dependencies lists it, if its sum wasn't captured then logrus could change a versioned release and increment a dependency you both share, thus breaking your reproducible build.Arabic
go.sum also contains versions that are not included in your build, so that that MVS can be verified.Maxa
P
9

Yes, it correct to say the modules really "used" by the application are listed by go list -m all (as per documentation you provided the link of). By "used", it means the package selected at build time for the compilation of the go code of your application.

We had a similar issue with a static analysis tool and we had to change the configuration to use the output of go list -m all (dumped in a file) instead of go.sum.

Personate answered 12/1, 2022 at 17:9 Comment(0)
S
1

No, go list -m all will most likely list a large superset of the modules used in the program itself. If you want to know what actually ends up being built into the program, build it and run go version -m <prog>. For example, given the following test.go:

package main

import (
        "fmt"

        "github.com/spf13/viper"
)

func main() {
        fmt.Println(viper.New())
}

go list -m all will show a rather large list of modules, representing everything that would be used if you actually exercised every feature and pulled in every corresponding package in github.com/spf13/viper.

On the other hand, running go build test.go and then go version -m test will show a much smaller list, representative of the small subset of github.com/spf13/viper and its corresponding dependencies that was actually used:

test: go1.22.1
        path    command-line-arguments
        dep     github.com/fsnotify/fsnotify    v1.7.0  h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
        dep     github.com/hashicorp/hcl        v1.0.0  h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
        dep     github.com/magiconair/properties        v1.8.7  h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
        dep     github.com/mitchellh/mapstructure       v1.5.0  h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
        dep     github.com/pelletier/go-toml/v2 v2.1.0  h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
        dep     github.com/sagikazarmark/slog-shim      v0.1.0  h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=
        dep     github.com/spf13/afero  v1.11.0 h1:WJQKhtpdm3v2IzqG8VMqrr6Rf3UYpEF239Jy9wNepM8=
        dep     github.com/spf13/cast   v1.6.0  h1:GEiTHELF+vaR5dhz3VqZfFSzZjYbgeKDpBxQVS4GYJ0=
        dep     github.com/spf13/pflag  v1.0.5  h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
        dep     github.com/spf13/viper  v1.18.2 h1:LUXCnvUvSM6FXAsj6nnfc8Q2tp1dIgUfY9Kc8GsSOiQ=
        dep     github.com/subosito/gotenv      v1.6.0  h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
        dep     golang.org/x/sys        v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
        dep     golang.org/x/text       v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
        dep     gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
        dep     gopkg.in/yaml.v3        v3.0.1  h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
        build   -buildmode=exe
        build   -compiler=gc
        build   CGO_ENABLED=1
        build   CGO_CFLAGS=
        build   CGO_CPPFLAGS=
        build   CGO_CXXFLAGS=
        build   CGO_LDFLAGS=
        build   GOARCH=amd64
        build   GOOS=linux
        build   GOAMD64=v1
Staghound answered 26/3, 2024 at 22:51 Comment(0)

© 2022 - 2025 — McMap. All rights reserved.