Flatten Dictionary with Helm

Question

Is there a way to flatten a dictionary with helm? I want to provide environment variables to the application from the chart by flattening a YAML config located in values.yaml. The config can look like. (Not Actual)

config:
 server:
  port: 3333
 other:
  setting:
    name: test

And would like to provide environment variables as

- name: CONFIG_SERVER_PORT
  value: 3333
- name: CONFIG_OTHER_SETTING_NAME
  value: test

I have considered using Kubernetes config maps but this would mean deploying slightly different instances of the app with random release names so that the config is not overwritten. This library https://github.com/jeremywohl/flatten provides a way to flatten a map[string]interface{} with delimeters. Is there a way to provide a custom pipe for helm that uses the library or another way to flatten the config?

Lashonlashond

Answer 1

I'm not aware of anything like that built in. Sprig provides most of the useful functions for helm templates but the dict functions just cover the primitives.

You could define a named template that does the business and recurses down the config dict/map. Then include the template where needed:

{{- define "recurseFlattenMap" -}}
{{- $map := first . -}}
{{- $label := last . -}}
{{- range $key, $val := $map -}}
  {{- $sublabel := list $label $key | join "_" | upper -}}
  {{- if kindOf $val | eq "map" -}}
    {{- list $val $sublabel | include "recurseFlattenMap" -}}
  {{- else -}}
- name: {{ $sublabel | quote }}
  value: {{ $val | quote }}
{{ end -}}
{{- end -}}
{{- end -}}

Passing the config data in is a little complex here, via a list that is then separated back out into $map and $label. This is due to templates only accepting a single variable scope.

env: {{ list .Values.config "CONFIG" | include "recurseFlattenMap" | nindent 2 }}   

With the example values:

config:
  server:
    port: 3333
  first: astr
  other:
    setting:
      name: test

Results in

$ helm template . 
---
# Source: so61280873/templates/config.yaml
env: 
  - name: "CONFIG_FIRST" 
    value: "astr"
  - name: "CONFIG_OTHER_SETTING_NAME" 
    value: "test"
  - name: "CONFIG_SERVER_PORT" 
    value: "3333"

Alcuin

Answer 2

Probably not. You might be able to implement this in pure Gotpl with some of the Sprig functions and a lot of local variables but ... don't. You cannot add custom functions to Helm without recompiling it. Just use the native format directly.

Activator

Is there a way then to write the subset of .Values on a yaml file and mount the file to a volume? – Lashonlashond

Yes, sprig has a toJson function (JSON is a subset of YAML) so you can use that in a configmap, then mount that configmap as a volume (can also use a secret, same difference). – Activator

Yes but i was trying to avoid config maps. The app can be configured to expose a public API or a private API. The private API is to be run as a sidecar available only to specific micro-services. If micro-service A wants a different config of the private API compared to micro-service B then in the end, with a few more deployments, there is a messy config list in kubernetes without knowing which config is for what side-car. Is there a way to mount a the file without a kubernetes resource? e.g data: | {{toYaml .Value.config}} – Lashonlashond

or rather without config maps but directly in the pod with the file data – Lashonlashond

Configmaps and secrets are how you expose config data as files to a pod. If you want to do that thing, then you need to use one of those (or, I guess, write your own CSI plugin). – Activator

Answer 3

probably not perfect:

{{- range $key, $value := (regexReplaceAll ":\n(  )+" (toYaml .Values) "_" | fromYaml) }}
{{ printf "- %s: %s" $key $value }}
{{- end }}

where .Values had the form

values:
- key:
    subKey: value
- key
    subKey2: value2

and I wanted

- key_subkey: value
- key2_subKey2: value2

specifically this came from using helmfile to manage several separate helm charts into one orchestrated command wanting to collapse the environment values into a configmap chart such that pods could envFrom.configMapRef to load the environment configuration in the helmfile as environment variables

Banditry

Answer 4

Another implementation, that I use:

inside the _helpers.tpl, create named template

{{- /*
  Flattens a yaml structure into a key-value dict. The keys become uppercase underscored version of the yaml structure.
  Example values:
    something:
      hello:
        world: 123
        universe: 456
      xy: abc
  
  Call: {{- $flattened := include "flattenYamlToUnderscored" (dict "prefix" "SOMETHING" "structure" $.Values.something) | fromYaml }}
  Returns: Key-Value-Map with flattened yaml, e.g.
    SOMETHING_HELLO_UNIVERSE: 456
    SOMETHING_HELLO_WORLD: 123
    SOMETHING_XY: abc
*/}}
{{- define "flattenYamlToUnderscored" -}}
  {{- $prefix := .prefix -}}
  {{- $structure := .structure -}}

  {{- if kindIs "map" $structure -}}
    {{/* This is a map, go on iterating over the children */}}
    {{- $result := dict -}}
    {{- range $key, $value := $structure -}}
      {{/* For each child generate the new key and call the recursion, and add its result to the result list */}}
      {{- $newKey := printf "%s_%s" $prefix $key | trimAll "_" | upper -}}
      {{- $childResult := (include "flattenYamlToUnderscored" (dict "prefix" $newKey "structure" $value) | fromYaml) -}}
      {{- $_ := merge $result $result $childResult -}}
    {{- end -}}
    {{/* Return the merged results of the children */}}
    {{- $result | toYaml -}}

  {{- else if not (eq $structure nil) -}}
    {{/* We are not a map, nor a nil, so we are a leaf inside the recursion. $structure is the leaf value. Return [$prefix : $structure] as result */}}
    {{- dict $prefix $structure | toYaml -}}
  {{- end -}}
{{- end -}}

Using the values

config:
  server:
    port: 3333
  other:
    setting:
      name: test

And then call it like this:

env:
{{- $flattenedValues := include "flattenYamlToUnderscored" (dict "prefix" "CONFIG" "structure" $.Values.config) | fromYaml }}
{{- range $key, $value := $flattenedValues }}
- name: {{ $key }}
  value: {{ $value | quote }}
{{- end }}

results in

env:
- name: CONFIG_OTHER_SETTING_NAME
  value: "test"
- name: CONFIG_SERVER_PORT
  value: "3333"

Justin

Answer 5

The question you are asking is possible. dont have sources near by. but try something like this.

// chart

apiVersion: apps/v1beta1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: {{ template "name" . }}          
          command: [{{ range $i, $e := .Values.container.command }}{{ if $i }}, {{$e|quote}}{{else}}{{$e|quote}}{{end}}{{end}}]          
          env:
          {{- range .Values.container.env }}
            - name: {{ .name }}
              value: "{{ .value }}"
          {{- end }}


// values

container:  
  command: ["cmd", "sub_cmd", "sub_sub_cmd"]    
  env:
      - name: CONFIG_SERVER_PORT
      value: 3333
      - name: CONFIG_OTHER_SETTING_NAME
      value: test

Ruderal

Also while it's entirely unrelated, the better way to do the command setting is command: {{ .Values.command | toJson }} :) – Activator