Reading in environmental Variable Using Viper Go

Question

I am trying to make Viper read my environment variables, but its not working. Here is my configuration:

# app.yaml
dsn: RESTFUL_APP_DSN
jwt_verification_key: RESTFUL_APP_JWT_VERIFICATION_KEY
jwt_signing_key: RESTFUL_APP_JWT_SIGNING_KEY
jwt_signing_method: "HS256"

And my config.go file:

package config

import (
    "fmt"
    "strings"

    "github.com/go-ozzo/ozzo-validation"
    "github.com/spf13/viper"
)

// Config stores the application-wide configurations
var Config appConfig

type appConfig struct {
    // the path to the error message file. Defaults to "config/errors.yaml"
    ErrorFile string `mapstructure:"error_file"`
    // the server port. Defaults to 8080
    ServerPort int `mapstructure:"server_port"`
    // the data source name (DSN) for connecting to the database. required.
    DSN string `mapstructure:"dsn"`
    // the signing method for JWT. Defaults to "HS256"
    JWTSigningMethod string `mapstructure:"jwt_signing_method"`
    // JWT signing key. required.
    JWTSigningKey string `mapstructure:"jwt_signing_key"`
    // JWT verification key. required.
    JWTVerificationKey string `mapstructure:"jwt_verification_key"`
}

func (config appConfig) Validate() error {
    return validation.ValidateStruct(&config,
        validation.Field(&config.DSN, validation.Required),
        validation.Field(&config.JWTSigningKey, validation.Required),
        validation.Field(&config.JWTVerificationKey, validation.Required),
    )
}

func LoadConfig(configpaths ...string) error {
    v := viper.New()
    v.SetConfigName("app")
    v.SetConfigType("yaml")
    v.SetEnvPrefix("restful")
    v.AutomaticEnv()
    v.SetDefault("error_file", "config/errors.yaml")
    v.SetDefault("server_port", 1530)
    v.SetDefault("jwt_signing_method", "HS256")
    v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))

    for _, path := range configpaths {
        v.AddConfigPath(path)
    }

    if err := v.ReadInConfig(); err != nil {
        return fmt.Errorf("Failed to read the configuration file: %s", err)
    }

    if err := v.Unmarshal(&Config); err != nil {
        return err
    }

    // Checking with this line. This is what I get:
    // RESTFUL_JWT_SIGNING_KEY
    fmt.Println("Sign Key: ", v.GetString("jwt_signing_key"))

    return Config.Validate()
}

This line fmt.Println("Sign Key: ", v.GetString("jwt_signing_key")) gives me just the key passed in the yaml file RESTFUL_JWT_SIGNING_KEY. I don't know what I'm doing wrong.

According to doc:

AutomaticEnv is a powerful helper especially when combined with SetEnvPrefix. When called, Viper will check for an environment variable any time a viper.Get request is made. It will apply the following rules. It will check for a environment variable with a name matching the key uppercased and prefixed with the EnvPrefix if set.

So, why isn't it reading the environment variables?

Using JSON

{
  "dsn": "RESTFUL_APP_DSN",
  "jwt_verification_key": "RESTFUL_APP_JWT_VERIFICATION_KEY",
  "jwt_signing_key": "RESTFUL_APP_JWT_SIGNING_KEY",
  "jwt_signing_method": "HS256"
}

And my parser looks like thus:

// LoadConfigEnv loads configuration from the given list of paths and populates it into the Config variable.
// Environment variables with the prefix "RESTFUL_" in their names are also read automatically.
func LoadConfigEnv(environment string, configpaths ...string) error {
  v := viper.New()
  v.SetConfigName(environment)
  v.SetConfigType("json")
  v.SetEnvPrefix("restful")
  v.AutomaticEnv()
  v.SetDefault("jwt_signing_method", "HS256")
  v.SetDefault("error_file", "config/errors.yaml")
  v.SetDefault("server_port", 1530)
  v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))

  for _, path := range configpaths {
    v.AddConfigPath(path)
  }

  if err := v.ReadInConfig(); err != nil {
    return fmt.Errorf("Failed to read the configuration file: %s", err)
  }

  if err := v.Unmarshal(&Config); err != nil {
    return err
  }

  return Config.Validate()
}

In the Validate function, I decided to check the Config struct, and this is what I got:

Config: {config/errors.yaml 1530 RESTFUL_APP_DSN HS256 RESTFUL_APP_JWT_SIGNING_KEY RESTFUL_APP_JWT_VERIFICATION_KEY}

Terryterrye

Cannot reproduce: play.golang.org/p/YvK__C7zx13 $ RESTFUL_JWT_SIGNING_KEY=from-env go run main.go outputs "Sign Key: from-env". – Pudgy

Add fmt.Printf("Getenv: %q\n", os.Getenv("RESTFUL_JWT_SIGNING_KEY")) to your program and let us know what that prints. – Pudgy

@Pudgy When I do that, it gets the Environment Variable's value Getenv: "QfCAH04Cob7b71QCqy738vw5XGSnFZ9d" – Terryterrye

How do you run it? Can you make sure you are using latest version of viper? – Nonflammable

@TarunLalwani Yes. I am. Check above, I decided to change it to JSON file, and still the same thing. – Terryterrye

Which OS are you using? – Nonflammable

@TarunLalwani I am using an Apple MacOS. Apparently, I have found the solution to my problem. This is actually an example of a vallue in my actual app.yaml file RESTFUL_APP_DSN a DSN value is like this: dsn: RESTFUL_APP_DSN so, I decided to to go through Viper codebase, and the decided to try app_dsn: RESTFUL_APP_DSN and Voila, it works fine. Which means, the Key must be the suffix of the prefix provided while setting up Viper – Terryterrye

That's a good find. You should post the details as an answer – Nonflammable

Done @TarunLalwani thanks. – Terryterrye

Answer 1

The code in my question didn't actually show what I was trying to solve. After understanding what was wrong, I believe, the code here would have ran locally in my Development Environment.

According to Viper documentation:

AutomaticEnv is a powerful helper especially when combined with SetEnvPrefix. When called, Viper will check for an environment variable any time a viper.Get request is made. It will apply the following rules. It will check for an environment variable with a name matching the key uppercased and prefixed with the EnvPrefix if set.

This line here says it all:

It will check for a environment variable with a name matching the key uppercased and prefixed with the EnvPrefix if set

The prefix set with v.SetEnvPrefix("restful") would be expecting a .yaml with key value of:

Example app.yaml:

dsn: RESTFUL_DSN

Notice the DSN being the Lowercased Key and it being used as a Suffix of RESTFUL_DSN

In my situation, I was doing this instead:

Example app.yaml:

dsn: RESTFUL_APP_DSN

So, it was checking for RESTFUL_DSN in my environment instead of RESTFUL_APP_DSN

Terryterrye