Helm: Executing command on other containers in Job

5/9/2019

I want to utilize Charts Hooks's post-install to do some action on my deployment's container.

For example, I have a php-fpm container that consists of a Laravel application, and I want to run php artisan key:gen on install. Since it's a one time command so I couldn't place it on the postStart lifecycle, otherwise it would keep overwriting the APP_KEY.

How can I use Charts Hooks to achieve it? Or is there a better way?

-- Cloud Soh Jun Fu
kubernetes
kubernetes-helm
laravel

3 Answers

5/9/2019

Your job needs to run a container that contains kubectl and you would execute this script to exec into another container. Since kubectl exec doesn't support selection by labels you need to retrieve the pod name beforehand:

$pod=$(kubectl get pods --no-headers -o custom-columns=":metadata.name" -l YOUR-LABELS=YOUR-VALUES)
kubectl exec $pod php artisan key:gen
-- Lukas Eichler
Source: StackOverflow

5/9/2019

If you think about the lifecycle of this key: if there are multiple pod replicas they need to agree on what the key is; and if you delete and recreate the pod, it needs to be using the same key it was using before. (A quick Google search comes up with some good descriptions of what this key is actually used for; if it's encrypting session cookies, for example, every copy of the pod really needs to agree.)

This suggests a setup where you generate the key once, store it in a Kubernetes Secret, and make it available to pods. Conveniently, "any variable in your .env file can be overridden by external environment variables", and you can set an environment variable from a secret value. There isn't a great way to make Helm generate the secret itself in a way that will be saved.

So, putting these pieces together: in your pod spec (inside your deployment spec) you need to get the environment variable from the secret.

env:
  - name: APP_KEY
    valueFrom:
      secretKeyRef:
        name: "{{ .Release.Name }}-{{ .Chart.Name }}"
        key: app-key

Then you need to create a secret to hold the key.

apiVersion: v1
kind: Secret
metadata:
  name: "{{ .Release.Name }}-{{ .Chart.Name }}"
data:
  app-key: {{ printf "base64:%s" .Values.appKey | b64enc }}

And finally create the file holding the key. This should not be checked in as part of your chart.

echo "appKey: $(dd if=/dev/urandom bs=32 count=1 | base64)" > values-local.yaml

When you go to install your chart, use this values file

helm install ./charts/myapp -f values-local.yaml

There are a couple of other reasonable approaches that involve injecting the whole .env file as a ConfigMap or Secret, or extending your Docker image to generate this file on its own from values that get passed into it, or using an init container to generate the file before the main container starts. The point is that pods come and go, and need to be able to configure themselves when they start up; using kubectl exec in the way you're suggesting isn't great practice.

-- David Maze
Source: StackOverflow

5/9/2019

You can define a Job that will be run only once when Helm chart is installed:

apiVersion: batch/v1
kind: Job
metadata:
  name: "{{.Release.Name}}"
  labels:
    app.kubernetes.io/managed-by: {{.Release.Service | quote }}
    app.kubernetes.io/instance: {{.Release.Name | quote }}
    app.kubernetes.io/version: {{ .Chart.AppVersion }}
    helm.sh/chart: "{{.Chart.Name}}-{{.Chart.Version}}"
  annotations:
    # This is what defines this resource as a hook. Without this line, the
    # job is considered part of the release.
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  template:
    metadata:
      name: "{{.Release.Name}}"
      labels:
        app.kubernetes.io/managed-by: {{.Release.Service | quote }}
        app.kubernetes.io/instance: {{.Release.Name | quote }}
        helm.sh/chart: "{{.Chart.Name}}-{{.Chart.Version}}"
    spec:
      restartPolicy: Never
      containers:
      - name: post-install-job
        image: "alpine:3.3"
        command: ["/bin/sleep","{{default "10" .Values.sleepyTime}}"]

If you want to run job every time you upgrade chart - you can specify "post-upgrade" hook. Read more here: https://github.com/helm/helm/blob/master/docs/charts_hooks.md

-- Vasily Angapov
Source: StackOverflow