HOCON array substitution from envs
Asked Answered
P

2

7

I am using the HOCON config format and parsing lib is from typesafe.Config. HOCON supports env vars injections and overrides. like:

my.config = "asd"
my.config = ${?MY_ENV_VAR} 

this will substitute the default value "asd" if there is a env var called MY_ENV_VAR presented. however I can't seem to find any good way to do list env substitution. like:

my.config = [1,2,3,4]
my.config = ${?MY_ENV_LIST}

because by default, env vars from outside will be default to string, so [1,2,3,4] will be considered as "[1,2,3,4]" from the library's perspective, therefore it can't be seen as a list and will produce runtime error like this:

com.typesafe.config.ConfigException$WrongType: application.conf: 5: application.boolliststring has type STRING rather than LIST

at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:133) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:145) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:151) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164) at com.typesafe.config.impl.SimpleConfig.getList(SimpleConfig.java:212)

anyone has any idea what is the proper way to do list env substitution in HOCON format? thanks in advance.

Palmate answered 7/3, 2019 at 19:50 Comment(2)
What does the env var MY_ENV_LIST contain ?Malfunction
@Malfunction MY_ENV_LIST='[1,2,3,4]', sourced from env vars. since env vars can only be in string type, therefore causing this problem.Palmate
U
11

The Answer

Going from one environment variable to an array is not possible out of the box, as the other answer states.

You could re-parse the string loaded from config:

// application.conf
my.config = "list = [1, 2, 3]"
my.config = ${?LIST_VAR}

// code
String listString = ConfigFactory.load().getString("my.config")
ConfigFactory.parseString(listString).getIntList("list")

Then set LIST_VAR='list = [4, 5, 6]' to override the defaults. Note that you need list = because an object is required at the top level of hocon, you can't use an array.

- or -

If your data is clean enough, you could just split on ,s:

// application.conf
my.config = "foo,bar,baz"
my.config = ${?CSV_VAR}

// java code
String csvString = ConfigFactory.load().getString("my.config")
String[] parameters = csvString.split(",")

Then, just set CSV_VAR=bing,bang,boom,pow (no []).

Further Reading

On the other hand, if you use separate environment variables for each value, there are several options.

No Defaults

The simplest, if you don't need defaults, looks like this:

my.config = [ ${?MY_ENV_VAR}, ${?MY_ENV_VAR_TWO} ]

Any values that are not defined are omitted.

Adding to defaults

If you only need to add to default values, you could use += syntax:

my.config = [1, 2]
my.config += ${?MY_ENV_VAR}
my.config += ${?MY_ENV_VAR_TWO}

Any values that are not defined are not added to the array.

Maximum Flexibility

The most flexible option I've found is to use positional syntax in your application.conf (or reference.conf or -D options or anywhere else you provide config):

my.config.0 = 1                    // always set to 1  
my.config.1 = 2                    // defaults to 2 if MY_ENV_VAR is not set
my.config.1 = ${?MY_ENV_VAR}
my.config.2 = ${?MY_ENV_VAR_TWO}   // totally optional
my.config.3 = ${MY_ENV_VAR_THREE}  // doesn't have ?, so it is required

Any values that are defined will be included, any that are not will be skipped over. For example, if MY_ENV_VAR=4, MY_ENV_VAR_THREE=6, and MY_ENV_VAR_TWO is not set, the resulting list will be [1, 4, 6].

List of objects

You can even define objects within the list, like this:

my.nested.0.myField = 1
my.nested.0.otherField = "hello"
my.nested.1.myField = 2
my.nested.1.myField = ${?MY_INT}
my.nested.1.otherField = "goodbye"
my.nested.1.otherField = ${?MY_STRING}
my.nested.2.myField = ${OTHER_INT}       // Note lack of ?
my.nested.2.otherField = ${OTHER_STRING} // Note lack of ?

One catch with a list of config objects, at least in my testing, is all items need to be completely defined. That is why the fields that don't have defaults are required substitutions. If MY_INT=99, MY_STRING is not set, OTHER_INT=100, and OTHER_STRING=foo, the above renders to:

other {
  nested = [
    { myField = 1, otherField = "hello" },
    { myField = 99, otherField = "goodbye" },
    { myField = 100, otherField = "foo" }
  ]
}
Unflinching answered 22/12, 2020 at 22:53 Comment(1)
Cool investigation! Thanks! Didn't know that HOCON is so powerfulLundt
C
1

No, that's not possible.

From Hocon docs:

environment variables always become a string value, though if an app asks for another type automatic type conversion would kick in

A note from same docs related to automatic type conversions:

The following type conversions should NOT be performed:

...

anything to array, with the exception of numerically-indexed object to array

Converting objects and arrays to and from strings is tempting, but in practical situations raises thorny issues of quoting and double-escaping.

Cheri answered 6/5, 2020 at 10:11 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.