K
Q

The node was low on resource: ephemeral-storage

1/25/2020

All the pods of a node are on Evicted state due to "The node was low on resource: ephemeral-storage."

portal-59978bff4d-2qkgf                            0/1     Evicted   0          14m
release-mgmt-74995bc7dd-nzlgq                      0/1     Evicted   0          8m20s
service-orchestration-79f8dc7dc-kx6g4              0/1     Evicted   0          7m31s
test-mgmt-7f977567d6-zl7cc                         0/1     Evicted   0          8m17s

anyone knows the quick fix of it.

-- Dasarathi Swain
kubernetes
kubernetes-pod

Similar Questions

What prevents the 'instance-groups' annotation from being set when creating a multi-cluster ingress on Google Kubernetes Engine?
kubernetes dashoboard 「Error trying to reach service: 'proxyconnect tcp: dial tcp: lookup $http_proxy: no such host」
GCE LoadBalancer not being created despite ingress resource creation
Inconsistent Behaviour When Writing to a File From Multiple Pods on an EFS Volume with ReadWriteMany
How to modify order of items in a WorkQueue before processing the items by worker thread of a k8s controller
Cronjob openshift not running a pod

1 Answer

1/25/2020

Pods that use emptyDir volumes without storage quotas will fill up this storage, where the following error is present:

eviction manager: attempting to reclaim ephemeral-storage

Set a quota limits.ephemeral-storage, requests.ephemeral-storage to limit this, as otherwise any container can write any amount of storage to its node filesystem.

A sample resource quota definition

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-resources
spec:
  hard:
    pods: "4" 
    requests.cpu: "1" 
    requests.memory: 1Gi 
    requests.ephemeral-storage: 2Gi 
    limits.cpu: "2" 
    limits.memory: 2Gi 
    limits.ephemeral-storage: 4Gi

Another reason for this issue can be log files eating disk space. Check this question

-- Arghya Sadhu
Source: StackOverflow