I am trying to deploy a Spring Boot application using configuration data from Kubernetes cluster. I have a simple RestController that prints a message by reading from a Kubernetes cluster.
private String message = "Message not coming from Kubernetes config map";
@RequestMapping(value="/echo", method=GET)
public String printKubeConfig() {
return message;
}
Specified the name of the config map in my application.yml
spring:
application:
name: echo-configmap
echo-configmap
apiVersion: v1
kind: ConfigMap
metadata:
name: echo-configmap
data:
application.properties: |-
message=Hello from dev Kubernetes Configmap
application_qa.properties: |-
message=Hello from qa Kubernetes Configmap
I have several environments like qa, int, test etc
Typically you want that applications running on qa
don't interfere with applications running on production
. Using Kubernetes, you can get this kind of isolation using different namespaces for different environments. That way, objects on the prod
namespace are different from objects on the qa
namespace. Another, more expensive approach would be to use different k8s clusters for different environments.
Having this setup, you would deploy your application in the namespace for the specific environment where you want to deploy to, creating the Deployment
object on that namespace. This Deployment
would make use of a ConfigMap
object containing your Spring Boot properties. Let's call this ConfigMap
echo-properties for example.
That way, every namespace would have a unique copy of the echo-properties ConfigMap
. Each containing the specific configuration for the environment where it belongs.
The Deployment
object consumes the ConfigMap
properties by either using environment variables or reading files. The important bit here is that if you change the echo-properties ConfigMap
data, your application won't see those new values, by default. Kubernetes doesn't have this feature so far. You can't compare ConfigMap
s to Spring Cloud Config, which is a dynamic configuration solution.
An approach that would get you a similar behaviour (but not quite the same) would be using the fabric8 ConfigMap Controller on your cluster. This controller is a process that runs on your cluster, and it would restart your application whenever the ConfigMap
changes, so that it reads the new configuration values.
If you don't want to restart your application whenever a configuration changes, you should probably stick to Spring Cloud Config for values that will potentially change, and use ConfigMaps
for other properties that won't change, like application name or port.
Your use case sounds very much like you should take look at spring-cloud-config
- https://cloud.spring.io/spring-cloud-config/
The config-server is an infrastructure component that serves configuration that could be located in a git repository.
A config-client application would connect to config-server
at startup and loads the configuration applicable to the current profiles.
You could have different branches for different environments - or use profiles per environment. In your kubernetes deployment manifest you could set the profile by setting SPRING_PROFILES_ACTIVE
environment variable.
This sounds like a good use case for Helm. You could deploy your application as a Helm Chart, which would basically allow you to generate your Kubernetes resources (like ConfigMaps, Deployments, and whatever else you need) from templates.
You can use the documentation on Helm Charts to get started with Helm. After having created a Chart with helm create
, you will get a templates/
directory, in which you might place the following YAML template for your ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ printf "%s-%s" .Release.Name .Chart.Name }}
labels:
app: {{ .Chart.Name | trunc 63 | trimSuffix "-" }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
data:
application.properties: |-
message={{ .Values.properties.message }}
You can add a second YAML template for your Deployment object (actually, helm create
will already create a sensible default deployment). Simply add your ConfigMap as volume, there:
containers:
- name: {{ .Chart.Name }}
# [...]
volumes:
- name: property-volume
mountPath: /etc/your-app/properties
volumes:
- name: property-volume
configMap:
name: {{ printf "%s-%s" .Release.Name .Chart.Name }}
Each Helm chart has a values.yaml
file, in which you can define default values that are then used to fill in your templates. This default file might look like this (remember that the ConfigMap template above contained a {{ .Values.properties.message }}
expression):
replicaCount: 1
image:
repository: your-docker-image
tag: your-docker-tag
properties:
message: Hello!
Next, use this Helm chart and the helm install
command to deploy your application as many times as you want with different configurations. You can supply different YAML files in which you override specific values from your values.yaml
file, or override individual values using --set
:
$ helm install --name dev --set image.tag=latest --set replicaCount=1 path/to/chart
$ helm install --name prod --set image.tag=stable --set replicaCount=3 --set properties.message="Hello from prod" path/to/chart
As to your second question: Of course you should put your Helm Chart into version control. You can then use the helm upgrade
command to apply changes to an already deployed application.
Let me try and provide an answer which I think gives you what you need, without using any tools beyond what you'll have installed on most boxes. Maybe try this first, and if you find the approach becomes difficult to manage and scale, move onto something more sophisticated.
Create a folder like k8s/configmaps
or something, and create one configmap per environment:
k8s/configmaps/properties.dev.yaml
k8s/configmaps/properties.qa.yaml
k8s/configmaps/properties.sit.yaml
k8s/configmaps/properties.uat.yaml
Each configmap should contain your environment specific settings.
Create a k8s namespace per environment, such as:
application-dev
application-qa
application-sit
application-uat
A little bash will help here:
#!/usr/bin/env bash
# apply-configmaps.sh
namespace="application-${ENVIRONMENT}"
for configmap in ./k8s/configmaps/*.${ENVIRONMENT}.yml; do
echo "Processing ConfigMap $configmap"
kubectl apply -n ${namespace} -f $configmap
done
Now all you need to do to create or update configmaps for any environment is:
ENVIRONMENT=dev ./update-configmaps.sh
Now you can create a CI/CD pipeline - if your configmap source changes just run the command shown above.
Based on primitive commands and no special tools you can:
I would strongly recommend you follow this basic 'first principles' approach before jumping into more sophisticated tools to solve the same problems, in many cases you can do it yourself without much effort, learn the key concepts and save the more sophisticated tooling till later if you really need it.
Hope that helps!
I would use this tool for each of your git projects for non secret data. https://github.com/kubernetes-sigs/kustomize
In the meta data you can filter your pods
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: mycomponent
env: dev
tier: backend
name: mycomponent
namespace: myapplication
kubectl get pods -n myapplication -l env=dev,tier=backend,app=mycomponent