"kubectl set image" fails with ErrImagePull

5/10/2017

Very often when I want to deploy new image with "kubectl set image" it's failing with ErrImagePull status, and then fixes itself after some time (up to few hours). These are events from "kubectl describe pod":

Events:
FirstSeen     LastSeen        Count   From                                                            SubObjectPath                   Type            Reason          Message
---------     --------        -----   ----                                                            -------------                   --------        ------          -------
36m           36m             1       {default-scheduler }                                                                            Normal          Scheduled       Successfully assigned zzz-staging-2373868389-62tgk to gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5
36m           12m             6       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}     spec.containers{zzz-staging}    Normal          Pulling         pulling image "us.gcr.io/yyyy-staging/zzz:latest"
31m           11m             6       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}     spec.containers{zzz-staging}    Warning         Failed          Failed to pull image "us.gcr.io/yyyy-staging/zzz:latest": net/http: request canceled
31m           11m             6       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}                                     Warning         FailedSync      Error syncing pod, skipping: failed to "StartContainer" for "zzz-staging" with ErrImagePull: "net/http: request canceled"

16m           7m              3       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}     spec.containers{zzz-staging}    Normal          BackOff         Back-off pulling image "us.gcr.io/yyyy-staging/zzz:latest"
16m           7m      7m      3       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}                                     Warning         FailedSync      Error syncing pod, skipping: failed to "StartContainer" for "zzz-staging" with ImagePullBackOff: "Back-off pulling image \"us.gcr.io/yyyy-staging/zzz:latest\""

24m           7m      5m      6       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}     spec.containers{zzz-staging}    Warning         InspectFailed   Failed to inspect image "us.gcr.io/yyyy-staging/zzz:latest": operation timeout: context deadline exceeded
24m           7m      5m      6       {kubelet gke-xxxxxxxxxx-staging-default-pool-ac6a32f4-09h5}                                     Warning         FailedSync      Error syncing pod, skipping: failed to "StartContainer" for "zzz-staging" with ImageInspectError: "Failed to inspect image \"us.gcr.io/yyyy-staging/zzz:latest\": operation timeout: context deadline exceeded"

Is there a way to avoid that?

-- Bart
google-kubernetes-engine
kubectl
kubernetes

0 Answers