I describe the pod. I found the error message. It seems the pod Readiness probe is failed. But I check the endpoint, the endpoint doesn't remove the pod IP. The pod receive traffic, even the liveness probe is failed.
Warning Unhealthy 92s (x84 over 3d1h) kubelet, aks-agentpool-xxxxx Readiness probe failed: Get https://10.244.14.21:5001/api/alive: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)$ kubectl describe ep myapp
Name: myapp
Namespace: default
Labels: <none>
Annotations: <none>
Subsets:
Addresses: 10.244.14.21,10.244.24.34
NotReadyAddresses: <none>
Ports:
Name Port Protocol
---- ---- --------
legacy 5001 TCP
https 5001 TCP
Events: <none>
I want to know what if readiness failed?
It seems the pod Readiness probe is failed. But I check the endpoint, the endpoint doesn't remove the pod IP.
The Pod is healthy at the time you are checking it and the Warning you see is just saying that there were situations when Readiness probe failed for it.
Please note that if the readiness probe fails, the endpoints controller removes the Pod’s IP address from the endpoints of all Services that match the Pod (they are added back upon Readiness probe success).
You can check that with $ kubectl describe svc/YOUR_SERVICE .
It'll output the list of EndPoints (EPs) that serve traffic for that service at a time you executed command.
If the EP readiness probe fails, EP won't be listed in the output of that command.
From the output you have provided:
$ kubectl describe ep myapp
...
Subsets:
Addresses: 10.244.14.21,10.244.24.34
NotReadyAddresses: <none>
we can say that all the EPs are ready at the moment.
I want to know what if readiness failed?
In short: traffic will be redirected to other EPs untill this EP is ready again.
Official documentation on Readiness probe says that:
A Probe is a diagnostic performed periodically by the kubelet on a Container. To perform a diagnostic, the kubelet calls a Handler implemented by the Container.
The kubelet can optionally perform and react to three kinds of probes on running Containers: - livenessProbe - readinessProbe - startupProbe
readinessProbe: Indicates whether the Container is ready to service requests. If the readiness probe fails, the endpoints controller removes the Pod’s IP address from the endpoints of all Services that match the Pod. The default state of readiness before the initial delay is Failure. If a Container does not provide a readiness probe, the default state is Success.
Hope that is exactly the info you have been looking for.