How does Kubernetes knows what pod to kill when downscaling?

9/10/2019

Is there a way to tell Kubernetes what pods to kill before or after a downscale? For example, suppose that I have 10 replicas and I want to downscale them to 5, but I want certain replicas to be alive and others to be killed after the downscale. Is that possible?

-- Matheus Melo
kubernetes

3 Answers

9/12/2019

you can use stateful sets instead of replicasets: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ they will be created sequentially (my-app0,my-app1,myapp2), and when you will scale down, they will be terminated in reverse order, from {N-1..0}.

-- iliefa
Source: StackOverflow

9/11/2019

While it's not possible to selectively choose which pod is killed, you can prevent what you're really concerned about, which is the killing of pods that are in the midst of processing tasks. This requires you do two things:

  1. Your application should be able to listen for and handle SIGTERM events, which Kubernetes sends to pods before it kills them. In your case, your app would handle SIGTERM by finishing any in-flight tasks then exiting.
  2. You set the terminationGracePeriodSeconds on the pod to something greater than the longest time it takes for the longest task to be processed. Setting this property extends the period of time between k8s sending the SIGTERM (asking your application to finish up), and SIGKILL (forcefully terminating).
-- Grant David Bachman
Source: StackOverflow

9/11/2019

As per provided by @Matt link and @Robert Bailey's answer, currently K8s ReplicaSets based resources don't support scaling functions, removing some specific Pods from replicas pool. You can find related #45509 issue and followed up #75763 PR.

-- mk_sta
Source: StackOverflow