I have some trouble with my k8s cluster, when I use kubectl set images to update my container in pod in order not to delete my pod.
Then, some ERRORS appear. The events in my dashboard show that the error is retried again and again and again.
Error syncing pod, skipping: failed to "KillContainer" for "usergraph1-test" with KillContainerError: "rpc error: code = 4 desc = context deadline exceeded"
The version of my Kubernetes is 1.7
here are my kubelet logs:
E0926 17:59:25.041040 18679 remote_runtime.go:224] StopContainer "751ba8d84ea5495987cef77ad3241f4753cf042689842966c2a97f439ed78728" from runtime service failed: rpc error: code
= 4 desc = context deadline exceeded
E0926 17:59:25.041279 18679 kuberuntime_container.go:611] Container "docker://751ba8d84ea5495987cef77ad3241f4753cf042689842966c2a97f439ed78728" termination failed with gracePer
iod 30: rpc error: code = 4 desc = context deadline exceeded
E0926 17:59:25.041362 18679 kuberuntime_manager.go:579] killContainer "usergraph1-test"(id={"docker" "751ba8d84ea5495987cef77ad3241f4753cf042689842966c2a97f439
ed78728"}) for pod "usergraph1-test_default(d8d1ab14-e03a-11e9-8496-a4dcbe00d64a)" failed: rpc error: code = 4 desc = context deadline exceeded
E0926 17:59:25.041444 18679 pod_workers.go:182] Error syncing pod d8d1ab14-e03a-11e9-8496-a4dcbe00d64a ("usergraph1-test_default(d8d1ab14-e03a-11e9-8496-a4dcbe
00d64a)"), skipping: failed to "KillContainer" for "usergraph1-test" with KillContainerError: "rpc error: code = 4 desc = context deadline exceeded"
I cannot find my docker daemon logs.
my docker version is :
Client:
Version: 1.12.6
API version: 1.24
Package version: docker-1.12.6-61.git85d7426.el7.centos.x86_64
Go version: go1.8.3
Git commit: 85d7426/1.12.6
Built: Tue Oct 24 15:40:21 2017
OS/Arch: linux/amd64
Server:
Version: 1.12.6
API version: 1.24
Package version: docker-1.12.6-61.git85d7426.el7.centos.x86_64
Go version: go1.8.3
Git commit: 85d7426/1.12.6
Built: Tue Oct 24 15:40:21 2017
OS/Arch: linux/amd64
Spending much time to find solutions makes me feel it is docker that cause this problem. But I don't know how to fix it.