I'm using Gitlab Autodevops to deploy app on my kubernetes cluster. That app should always have only one instance running. Problem is, during the update process, Helm kills currently running pod before the new pod is ready. This causes downtime period, when old version is already killed and new one isn't ready yet. To make it worse, app need significant time to start (2+ minutes).
I have tried to set minAvailable: 1
in PodDisruptionBudget
, but no help. Any idea how can i tell helm to wait for readiness of updated pod before killing old one? (Having 2 instances running simultaneously for several second is not such a problem for me)
You can release a new application version in few ways, it's necessary to choose the one that fit your needs. I would recommend one of the following:
Ramped - slow rollout
A ramped deployment updates pods in a rolling update fashion, a secondary ReplicaSet is created with the new version of the application, then the number of replicas of the old version is decreased and the new version is increased until the correct number of replicas is reached.
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 2 # how many pods we can add at a time
maxUnavailable: 0 # maxUnavailable define how many pods can be unavailable
# during the rolling update
Full example and steps can be found here.
Blue/Green - best to avoid API versioning issues
A blue/green deployment differs from a ramped deployment because the “green” version of the application is deployed alongside the “blue” version. After testing that the new version meets the requirements, we update the Kubernetes Service object that plays the role of load balancer to send traffic to the new version by replacing the version label in the selector field.
apiVersion: v1
kind: Service
metadata:
name: my-app
labels:
app: my-app
spec:
type: NodePort
ports:
- name: http
port: 8080
targetPort: 8080
# Note here that we match both the app and the version.
# When switching traffic, we update the label “version” with
# the appropriate value, ie: v2.0.0
selector:
app: my-app
version: v1.0.0
Full example and steps can be found here.
Canary - for testing
A canary deployment consists of routing a subset of users to a new functionality. In Kubernetes, a canary deployment can be done using two Deployments with common pod labels. One replica of the new version is released alongside the old version. Then after some time and if no error is detected, scale up the number of replicas of the new version and delete the old deployment.
Using this ReplicaSet technique requires spinning-up as many pods as necessary to get the right percentage of traffic. That said, if you want to send 1% of traffic to version B, you need to have one pod running with version B and 99 pods running with version A. This can be pretty inconvenient to manage so if you are looking for a better managed traffic distribution, look at load balancers such as HAProxy or service meshes like Linkerd, which provide greater controls over traffic.
Manifest for version A:
spec:
replicas: 3
Manifest for version B:
spec:
replicas: 1
Full example and steps can be found here.
You can also play with Interactive Tutorial - Updating Your App on Kubernetes.
I recommend reading Deploy, Scale And Upgrade An Application On Kubernetes With Helm.