Best practices for application configuration management in kubernetes and helm ecosystem

4/21/2020

Using kubernetes and helm for some time now, I find it quite unnatural to handle application configuration in kubernetes world with only helm.

A few reasons are:

  • Configuration is managed outside the cluster, using helm client and extra yamls for custom config. The extra values yaml files are not validated and could become quite large over time.
  • Extra values are not versioned nor persisted, you need to come up with your own way of keeping track of last applied custom configuration.
  • Configuration update is not user friendly: you'll have to update the extra values yaml, execute a helm upgrade, wait for your services to restart (and that's only if you properly wrote the helm chart to redeploy your workloads on config map updates). And that's only to update something like a threshold in your app.
  • Using etcd as a database for your app is more like an anti-pattern as etcd is already very sensitive to disk write latency due to its sequential IO requirements.
  • The reuse/reset values confusion helm reset reuse behavior

I was wondering if a hybrid solution like

  • use helm to only handle the kubernetes resources configuration
  • for application specific configuration use a centralized configuration service inside the cluster (e.g. something built on top of Spring Cloud Config or similar projects)

could be a better fit for this ecosystem.

How are you doing application level configuration management in k8s ? Are you happy with helm only ? Is there a better solution that you put in practice and it paid off ?

-- Laurentiu Soica
configuration
kubernetes
kubernetes-helm

0 Answers