This is a very wried thing.

I created a private GKE cluster with a node pool of 3 nodes. Then I have a replica set with 3 pods. some of these pods will be scheduled to one node.

So one of these pods always get ImagePullBackOff, I check the error

Failed to pull image "bitnami/mongodb:3.6": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

And the pods scheduled to the remaining two nodes work well.

I ssh to that node, run docker pull and everything is fine. I cannot find another way to troubleshoot this error.

I tried to drain or delete that node and let the cluster to recreate the node. but it is still not working.

Help me, please.

Update: From GCP documentation, it will fail to pull images from the docker hub.

BUT the weirdest thing is ONLY ONE node is unable to pull the images.