Difference between Priority and PriorityClass object in k8s

9/5/2019

I know what Priority class are in k8s but could not find anything about priority object after lot of searching.

So problem is I have created a PriorityClass object in my k8s cluster and set its value to -100000 and have created a pod with this priorityClass. Now when I do kubectl describe Pod I am getting two different field

Priority:           0

PriorityClassName:  imagebuild-priority

My admission-controller throws the following error

Error from server (Forbidden): error when creating "/tmp/tmp.4tJSpSU0dy/app.yml": 
pods "pod-name" is forbidden: the integer value of priority (0) must not be provided in pod spec; 
priority admission controller computed -1000000 from the given PriorityClass name

Somewhere it is setting Priority to 0 and PriorityClass trying to set it to -10000.

PriorityClass object has globalDefault: False

Command Run

kubectl create -f app.yml

Yaml file

---
apiVersion: v1
kind: Pod
metadata:
  name: image-builder-serviceacc
spec:
  securityContext:
    runAsUser: 0
  serviceAccountName: {{ serviceaccount }}
  automountServiceAccountToken: false
  containers:
  - name: container
    image: ....
    imagePullPolicy: Always
    env:
    - name: PATH
      value: "$PATH:/bin:/busybox/"
    command: [ "sh", "-c", "--" ]
    args: [ "while true; do sleep 30; done;" ]
  initContainers:
  - name: init-container
    image: ....
    imagePullPolicy: Always
    env:
    - name: PATH
      value: "$PATH:/bin:/busybox/"
    command: [ "sh", "-c", "--" ]
    args: [ "ls" ]
  restartPolicy: Always

Mutating controlled will append PriorityClass

-- prashant
kubernetes
pod-priority

1 Answer

9/5/2019

As per documentation:

PriorityClass also has two optional fields: globalDefault and description. The globalDefault field indicates that the value of this PriorityClass should be used for Pods without a priorityClassName. Only one PriorityClass with globalDefault set to true can exist in the system. If there is no PriorityClass with globalDefault set, the priority of Pods with no priorityClassName is zero.

This error means that u have collision

the integer value of priority (0) must not be provided in pod spec; 
priority admission controller computed -1000000 from the given PriorityClass name

You can fix it in 2 ways:

your should choose between globalDefault: true :

PriorityClass:

apiVersion: scheduling.k8s.io/v1beta1
kind: PriorityClass
metadata:
  name: high-priority-minus
value: -2000000
globalDefault: True
description: "This priority class should be used for XYZ service pods only."

Pod:

apiVersion: v1
kind: Pod
metadata:
  name: nginx5
  labels:
    env: test
spec:
  containers:
  - name: nginx5
    image: nginx
    imagePullPolicy: IfNotPresent
  priorityClassName: high-priority-minus 

priorityClassName can be used here, but you dont need to

Or with globalDefault: false :

You need to choose 1 option, priorityClassName or priority in your pod as described in you'r message error.

PriorityClass:

apiVersion: scheduling.k8s.io/v1beta1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000000
globalDefault: false
description: "This priority class should be used for XYZ service pods only."

Pod:

apiVersion: v1
kind: Pod
metadata:
  name: nginx7
  labels:
    env: test
spec:
  containers:
  - name: nginx7
    image: nginx
    imagePullPolicy: IfNotPresent
  priorityClassName: high-priority
-- jt97
Source: StackOverflow