generate dynamic runtime YAML for container with k8s secrets

11/26/2019

I'm deploying a service which rigidly takes a YAML config that looks something like this:

# /config/srv.yaml

#
# ...
#
tenantid: '...'
clientid: '...'
certfile: '/config/client/client.crt'
keyfile:  '/config/client/client.key'
#
# ...
#

TL;DR; I want to only store the tenantid, clientid & cert files in k8s secrets.

My current solution, stores the entire YAML file in secrets, which seems wasteful and also cumbersome to manage.


Essentially I have this - which works well for the 2 cert files (/config/client/client.crt and /config/client/client.key):

spec:
  containers:
    - name: my-container
      image: "my-image"
      imagePullPolicy: Always
      ports:
        - containerPort: 50100
      env:
      - name: CONF_FILE
        value: "/config/srv.yaml"
      volumeMounts:
      - name: yaml-vol
        mountPath: "/config" # KLUDGY
        readOnly: true
      - name: certs-vol
        mountPath: "/config/client"
        readOnly: true

    volumes:
    - name: yaml-vol
      secret:
        secretName: my-yamls # KLUDGY
    - name: certs-vol
      secret:
        secretName: my-certs # contains the *.crt/*.key cert files

However it involves storing the entire /config/srv.yaml in the kubernetes secrets my-yamls.

The k8s secrets docs suggest there's a way to create a dynamic YAML config for one's containers - filling in the secrets precisely where needed, using stringData e.g.

stringData:
  config.yaml: |-
    apiUrl: "https://my.api.com/api/v1"
    username: {{username}}
    password: {{password}}

but the docs trail off with a very vague:

Your deployment tool could then replace the {{username}} and {{password}} template variables before running kubectl apply.

I just need to fill in two string items in a dynamic config: clientid and tenantid.

Using just kubectl, how can one create a dynamic YAML for a container - storing the non-sensitive YAML template in the deploy.yaml - and have just the sensitive items in k8s secrets?

-- colm.anseo
kubectl
kubernetes
kubernetes-secrets

1 Answer

11/28/2019

An alternative will be to use another tool for secret management. One solution will be to use Kamus. Kamus support templating so you can do something like:

apiVersion: v1
kind: ConfigMap
metadata:
  name: encrypted-secrets-cm
data:
  tenantid: <encrypted>
  clientid: <encrypted>
  template.ejs: |
     tenantid: <%- secrets["tenantid"] %>
     clientid: <%- secrets["clientid"] %>
     certfile: '/config/client/client.crt'
     keyfile:  '/config/client/client.key'

Where the values are encrypted using Kamus.

And then either use clientSecret and store it the same way, or create a regular secret for both the crt and key. It's worth noticing that (assuming this is Azure) client id and tenant id are not considered secrets, and can be committed to a private repository.

Full disclosure: I'm Kamus author.

-- Omer Levi Hevroni
Source: StackOverflow