How to get the path to a Go module dependency?

Question

I have two Go modules, let's name them example.com/a and example.com/b.

Let this be example.com/a's go.mod:

module example.com/a

go 1.12

require (
  example.com/b v0.4.2
)

In example.com/b's root directory, there is a file named data.yaml. example.com/a needs to autogenerate some code as part of its build process. This autogeneration needs to read data.yaml.

How can I in the directory of example.com/a query for the path of example.com/b to read that file? I know that after downloading, the module will be somewhere in (go env GOPATH)/pkg/mod but I don't know how the path will be constructed from there as it contains some ! characters that are not part of the import path. I hoped that there is some subcommand of go mod or go list that will output the path, but I haven't found it in the documentation.

I have thought about including data.yaml in Go code via go-bindata (yes I'm aware of //go:embed but I don't want to require Go 1.16 for now) but then I would only have access at run-time when I need it at compile-time.

Thorfinn

Not sure if you considered this option but, you could have example.com/b export a function which uses runtime.Caller to figure out its file's location and using that constructs the absolute path to the yaml file and returns it. Then the example.com/a could invoke this function to get that location. I do use this approach myself where a imports b, a invokes a function of b, that function reads css/js/html files from its own module's folder hierarchy and then writes copies of those files into a. – Ingamar

@Ingamar Interesting idea, but still a run-time solution. The code generator runs before I have anything compiled and linked against example.com/b so I can't call a function from there. – Thorfinn

"but I don't know how the path will be constructed from there as it contains some ! characters that are not part of the import path." -- See the paragraph just above the table here: "To avoid ambiguity when serving from case-insensitive file systems, the $module and $version elements are case-encoded by replacing every uppercase letter with an exclamation mark followed by the corresponding lower-case letter. This allows modules example.com/M and example.com/m to both be stored on disk, since the former is encoded as example.com/!m." – Ingamar

On my system that turns the package github.com/BurntSushi/toml dir location into /Users/mkopriva/Work/go/pkg/mod/github.com/!burnt!sushi/[email protected] -- so maybe you can use that rule to make sure to find the package in the correct place. – Ingamar

Also, if your generator is written in Go, and you're ok with modifying it for this purpose, you could use the package golang.org/x/tools/go/packages to load a package with the packages.NeedModule flag, which will load info on the package's module, including it's directory, if there is any. – Ingamar

Answer 1

You can use go list with the -m flag and the -f flag like so:

go list -m -f '{{.Dir}}' example.com/b

The -m flag:

causes go list to list modules instead of packages. In this mode, the arguments to go list may be modules, module patterns (containing the ... wildcard), version queries, or the special pattern all, which matches all modules in the build list. If no arguments are specified, the main module is listed.

(reference)

The -f flag:

specifies an alternate format for the output, using the syntax of package template. The struct being passed to the template, when using the -m flag, is:

type Module struct {
    Path      string       // module path
    Version   string       // module version
    Versions  []string     // available module versions (with -versions)
    Replace   *Module      // replaced by this module
    Time      *time.Time   // time version was created
    Update    *Module      // available update, if any (with -u)
    Main      bool         // is this the main module?
    Indirect  bool         // is this module only an indirect dependency of main module?
    Dir       string       // directory holding files for this module, if any
    GoMod     string       // path to go.mod file for this module, if any
    GoVersion string       // go version used in module
    Error     *ModuleError // error loading module }

type ModuleError struct {
    Err string // the error itself
}

^{[the above quote was altered for context]}

(reference)

Ingamar

Though the packages.NeedModule solution would also be viable, this is certainly the easier solution. Thanks! – Thorfinn

Answer 2

You can figure out the module path like this:

package main

import (
    "fmt"
    "os"
    "path"

    "golang.org/x/mod/module"
)

func GetModulePath(name, version string) (string, error) {
    // first we need GOMODCACHE
    cache, ok := os.LookupEnv("GOMODCACHE")
    if !ok {
        cache = path.Join(os.Getenv("GOPATH"), "pkg", "mod")
    }

    // then we need to escape path
    escapedPath, err := module.EscapePath(name)
    if err != nil {
        return "", err
    }

    // version also
    escapedVersion, err := module.EscapeVersion(version)
    if err != nil {
        return "", err
    }

    return path.Join(cache, escapedPath+"@"+escapedVersion), nil
}

func main() {
    var path, err = GetModulePath("github.com/jakubDoka/mlok", "v0.4.7")
    if err != nil {
        panic(err)
    }

    if _, err := os.Stat(path); os.IsNotExist(err) {
        fmt.Println("you don't have this module/version installed")
    }
    fmt.Println("module found in", path)
}

Perspiratory

Good to know that the escaping routines used for creating the paths are exposed this way, thanks for showing that. I will go with mkopriva's solution since it automatically fetches the version number from go.mod which I would need to do manually here. – Thorfinn