I have the following pods on my kubernetes (1.18.3) cluster:

NAME      READY   STATUS    RESTARTS   AGE
pod1      1/1     Running   0          14m
pod2      1/1     Running   0          14m
pod3      0/1     Pending   0          14m
pod4      0/1     Pending   0          14m

pod3 and pod4 cannot start because the node has capacity for 2 pods only. When pod1 finishes and quits, then the scheduler picks either pod3 or pod4 and starts it. So far so good.

However, I also have a high priority pod (hpod) that I'd like to start before pod3 or pod4 when either of the running pods finishes and quits.

So I created a priorityclass can be found in the kubernetes docs:

kind: PriorityClass
metadata:
  name: high-priority-no-preemption
value: 1000000
preemptionPolicy: Never
globalDefault: false
description: "This priority class should be used for XYZ service pods only."

I've created the following pod yaml:

apiVersion: v1
kind: Pod
metadata:
  name: hpod
  labels:
    app: hpod
spec:
  containers:
  - name: hpod
    image: ...
    resources:
      requests:
        cpu: "500m"
        memory: "500Mi"
      limits:
        cpu: "500m"
        memory: "500Mi"
  priorityClassName: high-priority-no-preemption

Now the problem is that when I start the high prio pod with kubectl apply -f hpod.yaml, then the scheduler terminates a running pod to allow the high priority pod to start despite I've set 'preemptionPolicy: Never'.

The expected behaviour would be to postpone starting hpod until a currently running pod finishes. And when it does, then let hpod start before pod3 or pod4.

What am I doing wrong?