Kubernetes - Apply pod affinity rule to live deployment

1/23/2020

I am guess I am just asking for confirmation really. As had some major issues in the past with our elastic search cluster on kubernetes.

Is it fine to add a pod affinity to rule to a already running deployment. This is a live production elastic search cluster and I want to pin the elastic search pods to specific nodes with large storage. I kind of understand kubernetes but not really elastic search so dont want to cause any production issues/outages as there is no one around that could really help to fix it.

Currently running 6 replicas but want to reduce to 3 that run on 3 worker nodes with plenty of storage. I have labelled my 3 worker nodes with the label 'priority-elastic-node=true'

This is podaffinity i will add to my yaml file and apply:

   podAffinity:
         preferredDuringSchedulingIgnoredDuringExecution:
         - labelSelector:
             matchExpressions:
             - key: priority-elastic-node
               operator: In
               values:
                 - "true"
        topologyKey: "kubernetes.io/hostname"

What I assume will happen is nothing after I apply but then when I start scaling down the elastic node replicas the elastic nodes stay on the preferred worker nodes.

-- cjm888
elasticsearch
kubernetes

1 Answer

1/24/2020

Any change to the pod template will cause the deployment to roll all pods. That includes a change to those fields. So it’s fine to change, but your cluster will be restarted. This should be fine as long as your replication settings are cromulent.

-- coderanger
Source: StackOverflow