Running 1.11.2-gke.9 (COS image) I have installed installed gitlab-ci (including container registry) via helm chart. Everything is green.

Simple CI/Cd pipeline was pushing new images to gitlab docker registry. Push works.

On deployment - there is Error: ErrImagePull with

net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

After ssh'ing (gcp console ssh within browser) i noticed that the node can reach virtually the entire internet but not the very ingress the cluster is hosting.

Hence docker login/pull hanging.

How come that gitlab runner running inside GKE can push to registry, but the node that starts application pods cannot pull/login?

All FW rules are created by GKE itself, and they allow 80/443.

Routing quirk/bug?