K
Q

Install single Kubernetes deployment multiple times

1/3/2021

I have a Helm chart that installs different kubernetes resources to deploy my application. One of those resources is a deployment that has two flavors, one for a client part of the app and one for the server part, so actually they are two deployments. Most of their manifests (yaml files) are exactly the same, the only important difference is that each refers to a different configmap in order to have specific values for some of the configmap properties (particularly the type: client/server and number of replicas). This doesn't seem to be very efficient since I'm duplicating code for the deployments, but it's the way I found to do it. On the other hand, for the configmaps I made use of Helm's template feature ({{ include }}) so I have a "main" configmap template which has all the common content, and two separate configmaps specifying the differences for each deployment and including the main template.

So far so good, even though there may be some unnecessary code duplication, in which case I wouldn't know how to improve.

The problem is that multiple variants of the above two deployments came into play. For example, I may want to deploy a client-type pod with property X having a certain value, and two server-type pods with property X having a different value. So following my approach, I would have to start creating more deployment yaml files to cover all possible combinations: type=client & X=Y, type=client & X=Z, type=server & X=Y, type=server & X=Z and so on. And the only purpose of this is to be able to specify how many replicas I want for each kind or combination.

Is there any way (using Helm or other Kubernetes related framework) to have a single deployment yaml file and be able to install it multiple times specifying only the properties that vary and the number of replicas for that variation?

For example:

I want:

  • 3 replicas that have "type=client" and "X=1"
  • 2 replicas that have "type=server" and "X=1"
  • 4 replicas that have "type=client" and "X=2"

  • 1 replicas that have "type=server" and "X=3"

    where type and X are properties (data) in some configmap.

    Hope it's clear enough, otherwise please let me know, thanks.

-- Nicolás García
kubernetes
kubernetes-helm

Similar Questions

Kubernetes python eviction api complain about 'Name parameter required.'
Prevent Kubernetes breaking (kubectl does not respond) when too many Pods
spring boot usage of placeholder in application properties
Does Helm (or Kubernetes) caches charts?
How to use environment variables in init container args in kubernetes/openshift?

1 Answer

1/3/2021

In Helm there are a couple of ways to approach this. You need to bring the settings up to Helm's configuration layer (they would be in values.yaml or provided via a mechanism like helm install --set); you can't extract them out of the ConfigMap.

One approach is to have your Helm chart install only a single instance of the Deployment and the corresponding ConfigMap. Have a single templates/deployment.yaml file that includes lines like:

<!-- language: lang-yaml -->
name: {{ .Release.Name }}-{{ .Chart.Name }}-{{ .Values.type }}-{{ .Values.X }}

replicas: {{ .Values.replicas }}

env:
  - name: TYPE
    value: {{ .Values.type }}
  - name: X
    value: {{ quote .Values.X }}

Then you can deploy multiple copies of it:

<!-- language: lang-sh -->
helm install c1 . --set type=client --set X=1 --set replicas=3
helm install s1 . --set type=server --set X=1 --set replicas=2

You mention that you're generating similar ConfigMaps using templates already, and you can also use that same approach for any YAML structure. A template takes a single parameter, and one trick that's possible is to pass a list as that parameter. The other important detail to remember is that the top-level names like .Values are actually field lookups in a special object ., which can get reassigned in several contexts, so you may need to explicitly pass around and reference the top-level object.

Say your template needs the top-level values, and also some extra configuration settings:

<!-- language: lang-yaml -->
{{- define "a.deployment" -}}
{{- $top := index . 0 -}}
{{- $config := index . 1 -}}
metadata:
  name: {{ include "chart.name" $top }}-{{ $config.type }}-{{ $config.X }}
{{ end -}}

Note that we unpack the two values from the single list parameter, then pass $top in places where we might expect to pass . as a parameter.

You can have a top-level file per variant of this. For example, templates/deployment-server-1.yaml might contain:

<!-- language: lang-none -->
{{- $config := dict "type" "server" "X" "1" -}}
{{- include "a.deployment" (list . $config) -}}

Here . is the top-level object; we're embedding that and the config dictionary into a single list parameter to match what the template expects. You could use any templating constructs in the dict call if some of the values were specified in Helm configuration.

Finally, there's not actually a rule that a YAML file contains only a single object. If your Helm configuration just lists out the variants, you can loop through them and emit them all:

<!-- language: lang-none -->
{{-/* range will reassign . so save its current value */-}}
{{- $top := . -}}
{{- range .Values.installations -}}
{{-/* Now . is one item from the installations list */-}}
{{-/* This is the YAML start-of-document marker: */-}}
---
{{ include "a.deployment" (list $top .) -}}
{{- end -}}

You'd just list out all of the variants and settings in the Helm values.yaml (or, again, an externally provided helm install -f more-values.yaml file):

<!-- language: lang-yaml -->
installations:
  - type: client
    X: 1
    replicas: 3
  - type: server
    X: 1
    replicas: 2
-- David Maze
Source: StackOverflow