I have a bunch of images stored in GCR and deployed to a k8s cluster via helm charts. When i initiated a deployments the pod when into a failure saying imagepullbackoff. While the image was still there in GCR and is up and running in local. The k8s pods were down as it was unable to pull the images from GCR. Google storage status page shows no issues. After a brief amount of time, the pods started by pulling and building the very same image. Where should i look for logs for more information? Is GCR a single point of failure when it comes to auto scaling of pods?
The error message that you are getting “imagepullbackoff” is more related to the image pulling. Here is a post that explains how to troubleshoot “imagepullbackoff”.
You can run this command to get more information about a Pod’s container image:
When pulls are being slow you could ssh to node and run 'docker pull [image_name]', this will test performance of image pulling. Also you can use "imagePullPolicy" This could improve the performance.
When we talk about pods autoscaling we have 2 types : Horizontal Pod Autoscaler, and Vertical Pod Autoscaler So GCR is not a single point of failure when it comes to auto scaling of pods.
It may possible you have not added the secret in kubernetes so that it is not able to fetch the images from the GCR.
You can run kubectl describe pod <pod name> for more details and check for the reason of imagepullbackoff.
also check for any spelling mistake in image url of GCR.
it should also possible registry not found due to spelling mistake so check the describe events.