Helm template multi line secret values

12/3/2019

I have this helm template for secret object:

...
data:
{{- range $key, $val := fromYaml .Values.secretmap }}
  {{- $type := printf "%T" $val }}
  {{ $key }}: {{ if eq $type "float64"}}{{ printf "%.0f" $val | b64enc | quote }}{{ else }}{{ $val | b64enc | quote }}{{ end }}
{{- end }}
kind: Secret
...

And I load it as follows:

helm template --set-file secretmap="secretmap.yaml"

I create the secretmap.yaml from env var like so:

env | sed -n "s/^K8S_SECRET_\(.*\)$/\1/p" | sed s/=/': '/ \ >secretmap.yaml

The problem is with multiline values.
When I set a multi-line pem key as env var, only the first line inserted to the secretmap.yaml.

How can I load multi-line env var correctly to a yaml so helm could create a secret of it?

-- natdev
bash
kubernetes
kubernetes-helm

1 Answer

12/4/2019

I'd reach for a more powerful tool than a shell script to write out the secretmap.yaml file.

The Helm template itself looks fine. Assuming the content is valid YAML, it will echo it out, base64 encoding each value. You'd be happier if every node in the YAML were a string so you didn't have to reinterpret it based on a dynamic type lookup.

So the actual problem is generating the YAML file. You can take advantage of the fact that (a) YAML tries hard to make all valid JSON be valid YAML, and (b) almost every programming language includes JSON support in its standard library. (Or you can use Helm's fromJson function too.) Here's a minimal Python script that could write it out for you, in place of your sed command:

#!/usr/bin/env python3
import json
import os

PREFIX = 'K8S_SECRET_'
result = {}
for k in os.environ.keys():
    if k.startswith(PREFIX):
        kk = k[len(PREFIX):]
        v = os.environ[k]
        result[kk] = v
print(json.dumps(result))

Or, a denser one-liner based on jq

jq -n 'env | with_entries(select(.key | startswith("K8S_SECRET_")) | .key |= ltrimstr("K8S_SECRET_"))'

If the environment variables have newlines embedded in them, it's almost impossible to process this reliably with basic shell tools. As a minimal example try (bash/zsh specific syntax)

K8S_SECRET_FOO=$'bar\nK8S_SECRET_BAZ=quux' env

You want just one variable, but with the embedded newline, env will print this in a way indistinguishable from two separate variables.

-- David Maze
Source: StackOverflow