I'm starting to introduce liveness and readiness probes in my services, and I'm not sure if I've succeeded in getting it working or not, because I can't confidently interpret the status reported by kubectl.
kubectl describe pod mypod gives me something like this:
Name: myapp-5798dd798c-t7dqs
Namespace: dev
Node: docker-for-desktop/192.168.65.3
Start Time: Wed, 24 Oct 2018 13:22:54 +0200
Labels: app=myapp
pod-template-hash=1354883547
Annotations: version: v2
Status: Running
IP: 10.1.0.103
Controlled By: ReplicaSet/myapp-5798dd798c
Containers:
myapp:
Container ID: docker://5d39cb47d2278eccd6d28c1eb35f93112e3ad103485c1c825de634a490d5b736
Image: myapp:latest
Image ID: docker://sha256:61dafd0c208e2519d0165bf663e4b387ce4c2effd9237fb29fb48d316eda07ff
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Wed, 24 Oct 2018 13:23:06 +0200
Ready: True
Restart Count: 0
Liveness: http-get http://:80/healthz/live delay=0s timeout=10s period=60s #success=1 #failure=3
Readiness: http-get http://:80/healthz/ready delay=3s timeout=3s period=5s #success=1 #failure=3
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-gvnc2 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-gvnc2:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-gvnc2
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 84s default-scheduler Successfully assigned myapp-5798dd798c-t7dqs to docker-for-desktop
Normal SuccessfulMountVolume 84s kubelet, docker-for-desktop MountVolume.SetUp succeeded for volume "default-token-gvnc2"
Normal Pulled 75s kubelet, docker-for-desktop Container image "myapp:latest" already present on machine
Normal Created 74s kubelet, docker-for-desktop Created container
Normal Started 72s kubelet, docker-for-desktop Started container
Warning Unhealthy 65s kubelet, docker-for-desktop Readiness probe failed: Get http://10.1.0.103:80/healthz/ready: net/http: request canceled (Client.Timeout exceeded while awaiting headers)Now, I note that the container has Status: Ready, but the last event in the events list lists the state as Unhealthy because of a failed readiness probe. (Looking in the application logs I can see that there has been many more incoming requests to the readiness probe since, and that they all succeeded.)
How should I interpret this information? Does Kubernetes consider my pod to be ready, or not ready?
A pod is ready when readiness probes of all its containers return a success. In your case the readiness probe failed in first attempt but next probe was a success and the container went in ready state. Here in below example of failed readiness probe
the readiness probe below probed 58 times for last 11m and failed.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 11m default-scheduler Successfully assigned default/upnready to mylabserver.com
Normal Pulling 11m kubelet, mylabserver.com pulling image "luksa/kubia:v3"
Normal Pulled 11m kubelet, mylabserver.com Successfully pulled image "luksa/kubia:v3"
Normal Created 11m kubelet, mylabserver.com Created container
Normal Started 11m kubelet, mylabserver.com Started container
Warning Unhealthy 103s (x58 over 11m) kubelet, mylabserver.com Readiness probe failed: Get http://10.44.0.123:80/: dial tcp 10.44.0.123:80: connect: also the container status is not ready as can be seen below
kubectl get pods -l run=upnready
NAME READY STATUS RESTARTS AGE
upnready 0/1 Running 0 17mIn your case the readiness probe passed the health check and your pod is in ready state.
You can make use of initialDelaySeconds,periodSeconds,timeoutSeconds effectively to get better results. Here is a article.