What is the purpose of the `go mod vendor` command?

Question

The documentation says,

The go mod vendor command constructs a directory named vendor in the main module's root directory that contains copies of all packages needed to support builds and tests of packages in the main module. Packages that are only imported by tests of packages outside the main module are not included.

golangbyexample says:

You can also check in the vendor directory to your VCS (Version Control System). This becomes useful in sense that none of the dependency needs to be downloaded at run time as it is already present in the vendor folder checked into VCS

I think modules (go.mod,go.sum) take care of the versioning. I also think that dependencies are only downloaded when I run the program for the first time.

So, how is the command go mod vendor useful? What is its purpose or use case?

Plaice

When you vendor your dependencies, you can build the project without needing internet access (on another machine, for example) – Consols

When I run the program for the first time, the dependencies are downloaded in go/pkg/mod/cache/download/github.com and go/pkg/mod/github.com. To run it the second time, I do not need the internet connection anyways. So, how can "build the project without needing internet access" be the purpose of this command. What am I missing? – Plaice

It's often not your machine. But, say, a freshly started container in a CI cluster, which doesn't have internet. Also see Grokify's answer for another application/facet of this. – Consols

Answer 1

Go Modules takes care of versioning, but it doesn't necessarily take care of modules disappearing off the Internet or the Internet not being available. If a module is not available, the code cannot be built.

Go Proxy will mitigate disappearing modules to some extent by mirroring modules, but it may not do it for all modules for all time:

Why did a previously available module become unavailable in the mirror?

proxy.golang.org does not save all modules forever. There are a number of reasons for this, but one reason is if proxy.golang.org is not able to detect a suitable license. In this case, only a temporarily cached copy of the module will be made available, and may become unavailable if it is removed from the original source and becomes outdated. The checksums will still remain in the checksum database regardless of whether or not they have become unavailable in the mirror.

See more at: https://proxy.golang.org/

An alternative approach for this is to fork modules and use the Go Modules replace directive which allows redirecting an import path to another, e.g. your fork, in the go.mod file without changing your code. This approach was provided by colm.anseo.

Regarding Internet access, if you run a large server farm and need the code on multiple machines, downloading from the Internet to every machine in your farm can be inefficient and a security risk. It may be much more efficient to use go mod vendor into an internal repo and copy this around. Large companies use internal methods to deploy code to multiple servers in their data centers.

Steviestevy

So, it is like me cloning the project in my machine in case the developer deletes it. Is it a right statement? – Plaice

It's more like cloning the dependency project into your project (vs. machine). Once the code is in your project, you can commit it to your repo (which may not be accessible on the public Internet). This way you can have and manage a copy of the files yourself. – Steviestevy

You can also fork a copy of a critical 3rd party repo (if its no longer being maintained etc). Your imports can still refer to the internet version - but you can use the go.mod replace directive to redirect to your forked copy during builds. The replace directive can also point at local directories too. – Sd

Nice info @colm.anseo. I've added it to the answer and credited you. Thanks! – Steviestevy

You'll be able to use git diff to see the changes when you update a dependency, and this history will be maintained in your git repo. Source : honeybadger.io/blog/golang-go-package-management – Plaice

The use-cases I mentioned in golang.org/issue/33848 were “self-contained builds, language-agnostic code review, and ephemeral patches”. That's a lot more than just availability. – Mordacious

Great info @bcmills. I'll take a look and incorporate. – Steviestevy

That is a very indirect answer to say vendoring the deps is actually putting them with the project :) You basically say instead: it is taking them away from the internet. Ok cloud is the new local and vice versa I know ... :D – Harlamert

Answer 2

I also think that dependencies are only downloaded when I run the program for the first time

You have to first take into account, to run you first need to go build. Now go mod ... is a tool configuring where to put dependencies before the whole thing can be built. This however not by someone just interested to run it - but the developer of a program, be it for dev/test purpose or further deploy/ci etc.

It is mostly useful as it will allow to edit a dependency and rebuild the sourcetree with this modification. The default way of modules to be consumed by the compiler (as you may have seen) uses a user-home/global installation, where 1) it is not tied to the project 2) changes done there would have side-effects on other builds on the system sharing these deps eventually.

So, in a way it is really the only way to have a fully custom build and integral source-tree for your project, where any change in any file will reflect in the build and stay isolated however. Also, it is really to only way for me to envision to "debug" a dependency in the sense of changing the code or adding logs etc (disambiguation: not talking of using a debugger here).

Harlamert