We are currently testing different kubernetes configurations related to resilience nodeStatusUpdateFrequency, --node-monitor-grace-period, --leader-elect-lease-duration, etc.. To test these configurations we employ some automated testing, which has shown us that most of the times kubernetes reacts as expected.

The one test which is not reacting as expected is disabling the container runtime on a worker node (which simulates a container runtime failure).

In this tests it takes longer for the kubelet to recognize that the container runtime has failed than, for example, it takes to recognize a node failure. A node is regarded as Ready since the kubelet keeps posting a healthy nodeStatusUpdate. We can’t seem to find a configuration for the kubelet which influences how often the kubelet checks the health of the container runtime, is it possible to configure this?