K
Q

Manage multi-environment Kubernetes application on GKE?

6/10/2019

We are starting a project from scratch that will be managed on Google Cloud Services. I'd like to use Google Kubernetes Engine. Our application will have multiple environments (Dev, Staging, Production). Each environment is setup as a new Project on Google Cloud.

What is unclear to me is how to parameterize our service/manifest files. For instance our deploy file below, anything in {} I'd like to pull from a list of variables per environment. In a previous post someone mentioned using Helm, but I cannot find much documentation supporting the use of helm this way.

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: webapp
spec:
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: {max-surge}
      maxUnavailable: 0
  selector:
    matchLabels:
      run: webapp
  template:
    metadata:
      labels:
        run: webapp
    spec:
      containers:
      - name: webapp
        image: {gcr-image-url}
        imagePullPolicy: Always
        ports:
        - containerPort: 3000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: DATABASE_URL
        - name: SECRET_KEY_BASE
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: SECRET_KEY_BASE

What tools are available to manage my GKE environments? We'll use terraform for our infrastructure management, but again is there a larger wrapper I can use to set parameters per environment?

-- theartofbeing
devops-services
google-kubernetes-engine
kubernetes
kubernetes-helm

Similar Questions

Celery 'Error: No nodes replied within time constraint' error - how to debug?
How to configure Kubenetes to make external requests going from a static IP
Should jx step helm apply create/produce a helm release
Kafka Memory leak
Vuejs frontend gives 404 in kubernetes using nginx.ingress.kubernetes.io/rewrite-target annotation

2 Answers

6/10/2019

Helm would work for this, as would kustomize. In the case of helm, you'll have separate values.yaml files (e.g. dev-values.yaml) with e.g.:

max-surge: 2
gcr-image-url: project-23456/test

And then reference them in the yaml via:

{{ .Values.max-surge }}

The when installing you would use helm upgrade --install my-app . --values=dev-values.yaml

-- eamon1234
Source: StackOverflow
6/11/2019

https://get-ytt.io could be a solution.

Particularly if you look at this github discussion you will notice that you can configure your environment and then pass in values in the form of flags or environment variables.

In case of your example, given the following config.yml:

#@ load("@ytt:data", "data")

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: webapp
  annotations:
    environment: #@ data.values.env
spec:
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: #@ data.values.max_surge
      maxUnavailable: 0
  selector:
    matchLabels:
      run: webapp
  template:
    metadata:
      labels:
        run: webapp
    spec:
      containers:
      - name: webapp
        image: #@ data.values.gcr_image_url
        imagePullPolicy: Always
        ports:
        - containerPort: 3000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: DATABASE_URL
        - name: SECRET_KEY_BASE
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: SECRET_KEY_BASE

and values.yml:

#@data/values
---
env: staging
max-surge: 1
gcr-image-url: some/other-image:latest

assuming that everything is in the same directory, you can template config.yml like:

ytt -f .

or customize the values on the fly from env vars and command line arguments:

export CUSTOM_env=production
ytt -f . \
  --data-value max_surge=10 \
  --data-value gcr_image_url=some/image:1.0 \
  --data-values-env CUSTOM
-- nemo
Source: StackOverflow