Kubectl set images and end with "rpc error: code = 4 desc = context deadline exceeded"

9/26/2019

I have some trouble with my k8s cluster, when I use kubectl set images to update my container in pod in order not to delete my pod.

Then, some ERRORS appear. The events in my dashboard show that the error is retried again and again and again.

Error syncing pod, skipping: failed to "KillContainer" for "usergraph1-test" with KillContainerError: "rpc error: code = 4 desc = context deadline exceeded"

The version of my Kubernetes is 1.7

here are my kubelet logs:

E0926 17:59:25.041040   18679 remote_runtime.go:224] StopContainer "751ba8d84ea5495987cef77ad3241f4753cf042689842966c2a97f439ed78728" from runtime service failed: rpc error: code
 = 4 desc = context deadline exceeded
E0926 17:59:25.041279   18679 kuberuntime_container.go:611] Container "docker://751ba8d84ea5495987cef77ad3241f4753cf042689842966c2a97f439ed78728" termination failed with gracePer
iod 30: rpc error: code = 4 desc = context deadline exceeded
E0926 17:59:25.041362   18679 kuberuntime_manager.go:579] killContainer "usergraph1-test"(id={"docker" "751ba8d84ea5495987cef77ad3241f4753cf042689842966c2a97f439
ed78728"}) for pod "usergraph1-test_default(d8d1ab14-e03a-11e9-8496-a4dcbe00d64a)" failed: rpc error: code = 4 desc = context deadline exceeded
E0926 17:59:25.041444   18679 pod_workers.go:182] Error syncing pod d8d1ab14-e03a-11e9-8496-a4dcbe00d64a ("usergraph1-test_default(d8d1ab14-e03a-11e9-8496-a4dcbe
00d64a)"), skipping: failed to "KillContainer" for "usergraph1-test" with KillContainerError: "rpc error: code = 4 desc = context deadline exceeded"

I cannot find my docker daemon logs.

my docker version is :

Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-61.git85d7426.el7.centos.x86_64
 Go version:      go1.8.3
 Git commit:      85d7426/1.12.6
 Built:           Tue Oct 24 15:40:21 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-61.git85d7426.el7.centos.x86_64
 Go version:      go1.8.3
 Git commit:      85d7426/1.12.6
 Built:           Tue Oct 24 15:40:21 2017
 OS/Arch:         linux/amd64

Spending much time to find solutions makes me feel it is docker that cause this problem. But I don't know how to fix it.

-- love2k8s
docker
kubectl
kubelet
kubernetes

0 Answers