K
Q

Avoiding kubernetes scheduler to run all pods in single node of kubernetes cluster

6/13/2016

I have one kubernetes cluster with 4 nodes and one master. I am trying to run 5 nginx pod in all nodes. Currently sometimes the scheduler runs all the pods in one machine and sometimes in different machine.

What happens if my node goes down and all my pods were running in same node? We need to avoid this.

How to enforce scheduler to run pods on the nodes in round-robin fashion, so that if any node goes down then at at least one node should have NGINX pod in running mode.

Is this possible or not? If possible, how can we achieve this scenario?

-- Prakash
kubernetes
kubernetes-pod
scheduler

Similar Questions

unable to deploy kubernetes with DNS on a local ubuntu cluster (so one node)
Apache Mesos vs Google Kubernetes
Docker bootstrap unix sock is missing when reboot
kafka on kubernetes cannot produce/consume topics (ClosedChannelException, ErrorLoggingCallback)

3 Answers

4/1/2017

I think the inter-pod anti-affinity feature will help you. Inter-pod anti-affinity allows you to constrain which nodes your pod is eligible to schedule on based on labels on pods that are already running on the node. Here is an example.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    run: nginx-service
  name: nginx-service
spec:
  replicas: 3
  selector:
    matchLabels:
      run: nginx-service
  template:
    metadata:
      labels:
        service-type: nginx
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: service-type
                operator: In
                values:
                - nginx
            topologyKey: kubernetes.io/hostname
      containers:
      - name: nginx-service
        image: nginx:latest

Note: I use preferredDuringSchedulingIgnoredDuringExecution here since you have more pods than nodes.

For more detailed information, you can refer to the Inter-pod affinity and anti-affinity (beta feature) part of following link: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

-- Bruce Wu
Source: StackOverflow
3/18/2018

Use podAntiAfinity

Reference: Kubernetes in Action Chapter 16. Advanced scheduling

The podAntiAfinity with requiredDuringSchedulingIgnoredDuringExecution can be used to prevent the same pod from being scheduled to the same hostname. If prefer more relaxed constraint, use preferredDuringSchedulingIgnoredDuringExecution.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 5
  template:
    metadata:
      labels:                                            
        app: nginx                                   
    spec:
      affinity:
        podAntiAffinity:                                 
          requiredDuringSchedulingIgnoredDuringExecution:   <---- hard requirement not to schedule "nginx" pod if already one scheduled.
          - topologyKey: kubernetes.io/hostname     <---- Anti affinity scope is host     
            labelSelector:                               
              matchLabels:                               
                app: nginx        
      container:
        image: nginx:latest

Kubelet --max-pods

You can specify the max number of pods for a node in kubelet configuration so that in the scenario of node(s) down, it will prevent K8S from saturating another nodes with pods from the failed node.

-- mon
Source: StackOverflow
6/14/2016

The scheduler should spread your pods if your containers specify resource request for the amount of memory and CPU they need. See http://kubernetes.io/docs/user-guide/compute-resources/

-- DavidO
Source: StackOverflow