Kubernetes gives "use of closed network connection" failures on liveness and readiness checks once server gets load?

11/23/2015

I've introduced a bunch of readiness and liveness checks in our Kubernetes pods. And apart from currently being fairly CPU-heavy. They appear to work as expected.

But then we started to run some load-testing on our solution. And almost immediately pods gets killed and event like this show up:

Liveness probe errored: read tcp 10.244.27.123:8080: use of closed network connection

There appear to have been an issue with keep-alive and the http probe (issue 15643). But that also appears to have been fixed by disabling keep-alive in the probe in Kubernetes 1.1.1 (which is what we are running)

So does anyone have any idea what could be going on?

-- Kristoffer
kubernetes

1 Answer

1/17/2016

I have seen this error when the liveness probe is timing out. Try lengthening the timeoutSeconds on your livenessProbe and see if the problem goes away.

-- Ian Lewis
Source: StackOverflow