I created a Go client that connects to a server, rpc exchange, log and exit. Deploying in kubernetes, I expected this to move from "Running -> Completed -> Running" loop, similar to how dnsutils is deployed with a "sleep 3600" command to run for an hour and exit.
I reduced this to the smallest go program to make sure that no complexity intrudes, and this is:
k8s-tester ➜ cat client.go
package main
func main() {
return
}
When I this though, I actually see it go through "Running -> Completed -> CrashLoopBackOff -> Running" states.
Q: Why is kubernetes seeing this program as crashed when it, when run from the shell, shows an exit status of 0 (EXIT_SUCCESS)?
k8s-tester ➜ kubectl create deployment --image kvaradha/client test-client deployment.apps/test-client created k8s-tester ➜ kubectl get pods -w -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES hello-world-server-85486b969b-x744z 1/1 Running 0 10h 10.1.0.159 docker-desktop
linux-544887f4fb-gksd8 1/1 Running 12 14h 10.1.0.156 docker-desktop
postgres-57944b47cb-2cqtk 1/1 Running 0 13d 10.1.0.136 docker-desktop
test-client-74cddbb988-rxx8f 0/1 Completed 1 4s 10.1.0.169 docker-desktop
test-client-74cddbb988-rxx8f 0/1 CrashLoopBackOff 1 5s 10.1.0.169 docker-desktop
test-client-74cddbb988-rxx8f 0/1 Completed 2 21s 10.1.0.169 docker-desktop
test-client-74cddbb988-rxx8f 0/1 CrashLoopBackOff 2 33s 10.1.0.169 docker-desktop
test-client-74cddbb988-rxx8f 0/1 Completed 3 50s 10.1.0.169 docker-desktop
test-client-74cddbb988-rxx8f 0/1 CrashLoopBackOff 3 62s 10.1.0.169 docker-desktop
More details:
Simple Dockerfile that I used:
FROM scratch
MAINTAINER "kannan@ieee.org"
ARG ARCH
COPY client /client
ENTRYPOINT [ "/client" ]Build and deploy as:
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o client .
docker build -t kvaradha/client .
kubectl create deployment --image kvaradha/client test-clientOutput of docker, suggested by @sfrehse:
k8s-tester ➜ docker run --rm kvaradha/client 2020/02/28 21:33:17 hostname: 127.0.0.1:65432 name: H809430 k8s-tester ➜ echo $? 0
@CurtisMatton: describing the pod gets me:
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m2s default-scheduler Successfully assigned default/test-client-74cddbb988-jmxmd to docker-desktop Warning Failed 110s kubelet, docker-desktop Failed to pull image "kvaradha/client": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout Warning Failed 110s kubelet, docker-desktop Error: ErrImagePull Normal BackOff 109s kubelet, docker-desktop Back-off pulling image "kvaradha/client" Warning Failed 109s kubelet, docker-desktop Error: ImagePullBackOff Normal Pulling 55s (x4 over 2m1s) kubelet, docker-desktop Pulling image "kvaradha/client" Normal Created 54s (x3 over 2m) kubelet, docker-desktop Created container client Normal Pulled 54s (x3 over 2m) kubelet, docker-desktop Successfully pulled image "kvaradha/client" Normal Started 53s (x3 over 2m) kubelet, docker-desktop Started container client Warning BackOff 9s (x7 over 96s) kubelet, docker-desktop Back-off restarting failed container
Built a container with "FROM golang", same end result:
test-client-bd4c475d4-cmnd6 1/1 Running 2 30s test-client-bd4c475d4-cmnd6 0/1 Completed 2 31s hello-world-client-77775898c-564lt 0/1 Completed 325 27h test-client-bd4c475d4-cmnd6 0/1 CrashLoopBackOff 2 44s hello-world-client-77775898c-564lt 0/1 CrashLoopBackOff 325 27h
This is an intended and expected behavior. Kubernetes main job is to ensure that a given application is running at all times. If an application exits (regardless of the exit code), Kubernetes (more specifically the kubelet daemon) will assume that the application has crashed and will start it again. Hence it is expected of your application to be a long running process (I.e.: a server).
If your application is meant to run only once and then cleanly exit, a Kubernetes Job would be more appropriate instead of a Deployment. That way the pod will run to completion with no CrashLoopBackOff errors.