I have a question about K8S pod anti affinity rule. What I have and what I need.
I have cross 2 data centers K8S cluster. In every DC is for example 5 nodes. I have a Deployment for Pod, which runs in 10 replicas cross all 10 nodes. For every node is 1 Pod replica. And I want to set up rule for case, if one DC will crash, to not migrate 5 replicas from crashed DC to health DC.
I found, that it could be possible to do it throught "Anti-affinity" rule, but I can't find any example for this scenario. Do you have example for it?
From the documentation https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
you need to set a selector on your deployment and indicate in the anti-affinity section what is the value to match and make the anti-affinity true:
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis-cache
spec:
selector:
matchLabels:
app: store
replicas: 3
template:
metadata:
labels:
app: store
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- store
topologyKey: "kubernetes.io/hostname"
containers:
- name: redis-server
image: redis:3.2-alpineYou can see that it is using a label selector that try to find the key app with value store it means that if a node has already a pod with that label and value kubernetes will apply anti-affinity.
look at DaemonSet. It will deploy one replica on each node. If one DC is crashed then the pods will not be redeployed to other DC.