Best practice for the generation of in-repo protos when using Go Modules

Question

tl;dr A repo formerly configured to use GOPATH is now configured for Modules. All's good and better. However, protoc correctly (!) generates Golang code for protobufs defined within the repo in a github.com/path/to/repo/protos structure when I'd now prefer these to be generated in my sources, outside of GOPATH. I'm moving them to resolve this. Is there a better solution?

---

I have a GitHub repo. For the sake of discussion, let's call it github.com/acme/toolbox. In a subdirectory, I have protobuf files that include:

package acme.toolbox.v1;
option go_package = "github.com/acme/toolbox/protos";

When I was GOPATH'ing, all was well and protoc would generate Golang bindings in $GOPATH/src/github.com/acme/toolbox/protos and my code, importing pb "github.com/acme/toolbox/protos", would work.

Moving to Go Modules hasn't been pain-free but, the benefits outweigh the cost and I'm future-proofing myself and the code.

My issue is that I don't see how I can get protoc to generate the Golang bindings into my arbitrarily and outside of GOPATH located clone.

I'm moving the files after they're generated but this feels... inelegant:

cd ${TOOLBOX}
protoc \
--proto_path=./protos \
--go_out=plugins=grpc:/go/src
./protos/*.proto
mv ${GOPATH}/src/github.com/acme/toolbox/protos/*.go ${TOOLBOX}/protos

Is there a better solution?

Interlard

Answer 1

The main point of the go_package option is to define what the go package name will be. With that said, it can behave differently depending on what you set it too.

If option go_package is defined to be a valid go package name (e.g. protos), protoc will generate the files in the folder defined by --go_out with that package name. If option go_package is instead a path (e.g. github.com/acme/toolbox/protos), protoc will create the folder structure defined relative to --go_out and place the files there with the package name being the same as the last folder name.

Unless I am mistaken in what you are wanting to do, you can change go_package to be:

option go_package = "protos";

and change your protoc invocation to be:

protoc \
--proto_path=./protos \
--go_out=plugins=grpc:${TOOLBOX}/protos
./protos/*.proto

Doing that, the generated files will be placed in ${TOOLBOX}/protos with the go package package protos.

Kimbrakimbrell

Thank you. I think this approach would work. However, Go Modules provides the ability to define my project as e.g. github.com/acme/toolbox even though the files are stored in some arbitrary directory. This is because go.mod includes a module prefix that references that github path. If -- and perhaps this is my mistake -- protos is a subdirectory of the project, Go files in my project should refer to its files using the module prefix. – Interlard

On reflection, this is exactly the correct answer. For some (stupid) reason, I was not thinking through the consequences of using go_package using a ${GOPATH} value. You're, of course, correct. I revised go_package to be protos and the bindings are generated where I want them. Thank you! – Interlard

Thank you Jacob. The first sentence saved my life. Google needs to explain things in a simple way... – Mitchmitchael