As far as I can tell, if a Kubernetes pod fills an emptyDir volume that has a sizeLimit, the pod is immediately evicted, without following normal termination procedure. (terminationGracePeriodSeconds, et cetera)
Is there a way to set a soft eviction threshold for an emptyDir volumes so that it does follow normal termination procedures?
If not, does anybody have a good suggestion on how to build an appropriate livenessProbe? df shows disk free for the underlying disk, not the emptyDir, and du can be quite expensive.
You can tweak your soft eviction thresholds for the kubelet
There are 3 parameters eviction-soft, eviction-soft-grace-period, and eviction-max-pod-grace-period
More info here