lately I started using private repositories in docker-hub with my kubernetes cluster.
my kubernetes cluster version is 1.9.2 ( kubelet , kubeadm , kubectl )
my kubernetes cluster has about 12 deployments and they are all in default namespace.
I also created a docker registry secret:
kubectl create secret docker-registry registrykey ..
and I patched it into default namespace:
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "registrykey"}]}'
all my deployments works perfectly except one which keeps giving me ImagePullBackOff
when I try to describe it I get:
Normal BackOff 29m (x6112 over 1d) kubelet, node1 Back-off pulling image "registry.hub.docker.com/my-image"
Warning Failed 19m (x6155 over 1d) kubelet, node1 Error: ImagePullBackOff
Warning Failed 14m (x287 over 1d) kubelet, node1 Failed to pull image "registry.hub.docker.com/my-image": rpc error: code = Unknown desc = Error response from daemon: repository registry.hub.docker.com/my-image not found: does not exist or no pull access
Normal Pulling 3m (x289 over 1d) kubelet, node1 pulling image "registry.hub.docker.com/my-image"
I tried adding to kubelet service the following flags:
--image-pull-progress-deadline=5m --serialize-image-pulls=false
I also added docker service the following flag: --max-concurrent-downloads=10
the pull still fails.
the size of the image is 1.56 giga.
if I pull it manually via docker pull and change imagePullPolicy: Always
to imagePullPolicy: IfNotPresent
, it works. since I don't want to do it manually every deployment I am stuck.
any idea why it is happening and how to solve it will be welcomed.
many thanks
edit: added service account yaml:
apiVersion: v1
imagePullSecrets:
- name: registrykey
kind: ServiceAccount
metadata:
creationTimestamp: 2018-09-04T06:55:04Z
name: default
namespace: default
resourceVersion: "11698"
selfLink: /api/v1/namespaces/default/serviceaccounts/default
uid: 73c55e52-b00f-11e8-afec-0050569e63f0
secrets:
- name: default-token-ptx9l
and secret:
apiVersion: v1
data:
.dockerconfigjson: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
kind: Secret
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{".dockerconfigjson":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"},"kind":"Secret","metadata":{"annotations":{},"creationTimestamp":null,"name":"registrykey","namespace":"default"},"type":"kubernetes.io/dockerconfigjson"}
creationTimestamp: 2018-09-04T08:41:40Z
name: registrykey
namespace: default
resourceVersion: "11689"
selfLink: /api/v1/namespaces/default/secrets/registrykey
uid: 57ebf775-b01e-11e8-a16b-0050569e63f0
type: kubernetes.io/dockerconfigjson
I am using a docker hub private repositories
I am still desperate. has any one ever encountered this behavior before?