I have installed K8S with kubeadm and docker 18.09.4 and it works fine. Then I installed gcloud, ran gcloud init and select my project where gcr is activated, continued with gcloud components install kubectl docker-credentials-gcr, followed by docker-credentials-gcr configure-docker.

At that stage, docker can pull images from my own gcr registry, while kubelet cannot.

Basically, if I run docker run --rm --name hello gcr.io/own-gcr/hello-world it pulls the image from the registry and starts the container. If I delete the image from my local registry and ran `` it fails with the following description:

  Events:
  Type     Reason     Age                From                       Message
  ----     ------     ----               ----                       -------
  Normal   Scheduled  23s                default-scheduler          Successfully assigned default/node-hello-6b99957775-9dvvw to lfr025922-docker
  Normal   BackOff    20s (x2 over 21s)  kubelet, lfr025922-docker  Back-off pulling image "gcr.io/own-gcr/node-hello"
  Warning  Failed     20s (x2 over 21s)  kubelet, lfr025922-docker  Error: ImagePullBackOff
  Normal   Pulling    9s (x2 over 22s)   kubelet, lfr025922-docker  Pulling image "gcr.io/own-gcr/node-hello"
  Warning  Failed     9s (x2 over 21s)   kubelet, lfr025922-docker  Failed to pull image "gcr.io/own-gcr/node-hello": rpc error: code = Unknown desc = Error response from daemon: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
  Warning  Failed     9s (x2 over 21s)   kubelet, lfr025922-docker  Error: ErrImagePull

I of course, followed all instructions on page https://cloud.google.com/container-registry/docs/advanced-authentication and none of them were successful.

Are you aware of any issue with kubelet 1.14 and docker 18.09.5? Isn't kubelet supposed to rely on the underlying CRI (here docker)? Have you any idea of what could cause that issue?