I understand the concept of setting a request
or a limit
on a Kubernetes pod for both CPU
and/or memory
resources but I'm trying to understand what happens if you don't set either request
or limit
for say a CPU?
We have configured an NGINX pod but it doesn't have either a request
or limit
set for its CPU
. I'm assuming it will have at a minimum 1 millicore
and will give as much millicores to that pod as it needs and is available on the node. If the node has exhausted all available cores, then does it just stay stuck at 1 millicore?
What happens if you don't set either request or limit for say a CPU?
When you don’t specify a request for CPU, you’re saying you don’t care how much CPU time the process running in your container is allotted.
In the worst case, it may not get any CPU time at all (this happens when a heavy demand by other processes exists on the CPU). Although this may be fine for low-priority batch jobs, which aren’t time-critical, it obviously isn’t appropriate for containers handling user requests.
you’re also requesting
1 millicore
of memory for the container. By doing that, you’re saying that you expect the processes running inside the container to use at mostN mebibytes
of RAM. They might use less, but you’re not expecting them to use more than that in normal circumstances.
Understanding how resource requests affect scheduling
By specifying resource requests, you’re specifying the minimum amount of resources your pod needs. This information is what the Scheduler uses when scheduling the pod to a node.
Each node has a certain amount of CPU and memory it can allocate to pods. When scheduling a pod, the Scheduler will only consider nodes with enough unallocated resources to meet the pod’s resource requirements.
If the amount of unallocated CPU or memory is less than what the pod requests, Kubernetes will not schedule the pod to that node, because the node can’t provide the minimum amount required by the pod.
Understanding what will happened if Exceeding the limits
With CPU
CPU is a compressible resource, and it’s only natural for a process to want to consume all of the CPU time when not waiting for an I/O operation.
a process’ CPU usage is throttled, so when a CPU limit is set for a container, the process isn’t given more CPU time than the configured limit.
With Memory
With memory, it’s different. When a process tries to allocate memory over its limit, the process is killed (it’s said the container is
OOMKilled
, whereOOM
stands for Out Of Memory). If the pod’s restart policy is set to Always orOnFailure
, the process is restarted immediately, so you may not even notice it getting killed. But if it keeps going over the memory limit and getting killed, Kubernetes will begin restarting it with increasing delays between restarts. You’ll see aCrashLoopBackOff
status in that case.
kubectl get po
NAME READY STATUS RESTARTS AGE
memoryhog 0/1 CrashLoopBackOff 3 1m
Note: The CrashLoopBackOff
status doesn’t mean the Kubelet has given up. It means that after each crash, the Kubelet is increasing the time period before restarting the container.
Understand To examine why the container crashed
kubectl describe pod
Name:
...
Containers:
main: ...
State: Terminated
Reason: OOMKilled
...
Pay attention to the Reason attribute
OOMKilled
. The current container was killed because it was out of memory (OOM).