Error from server (BadRequest): container "espace-client-client" in pod "espace-client-client" is waiting to start: trying and failing to pull image

9/23/2019

I've deployed my first app on my Kubernetes prod cluster a month ago.

I could deploy my 2 services (front / back) from gitlab registry.

Now, I pushed a new docker image to gitlab registry and would like to redeploy it in prod:

Here is my deployment file:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    reloader.stakater.com/auto: "true"
  labels:
    app: espace-client-client
  name: espace-client-client
  namespace: espace-client
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      labels:
        app: espace-client-client
    spec:
      containers:
      - envFrom:
        - secretRef:
           name: espace-client-client-env
        image: registry.gitlab.com/xxx/espace_client/client:latest
        name: espace-client-client
        ports:
        - containerPort: 3000
        resources: {}
      restartPolicy: Always
      imagePullSecrets:
        - name: gitlab-registry

I have no clue what is inside gitlab-registry. I didn't do it myself, and the people who did it left the crew :( Nevertheless, I have all the permissions, so, I only need to know what to put in the secret, and maybe delete it and recreate it.

It seems that secret is based on my .docker/config.json

➜  espace-client git:(k8s) ✗ kubectl describe secrets gitlab-registry                                                                                
Name:         gitlab-registry
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/dockerconfigjson

Data
====
.dockerconfigjson:  174 bytes

I tried to delete existing secret, logout with

docker logout registry.gitlab.com
kubectl delete secret gitlab-registry

Then login again:

docker login registry.gitlab.com -u myGitlabUser
Password: 
Login Succeeded

and pull image with:

docker pull registry.gitlab.com/xxx/espace_client/client:latest

which worked.

file: ~/.docker/config.json is looking weird:

{
        "auths": {
                "registry.gitlab.com": {}
        },
        "HttpHeaders": {
                "User-Agent": "Docker-Client/18.09.6 (linux)"
        },
        "credsStore": "secretservice"
}

It doesn't seem to contain any credential...

Then I recreate my secret

kubectl create secret generic gitlab-registry \
    --from-file=.dockerconfigjson=/home/julien/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson

I also tried to do :

kubectl create secret docker-registry gitlab-registry --docker-server=registry.gitlab.com --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>

and deploy again:

kubectl rollout restart deployment/espace-client-client -n espace-client

but I still have the same error:

Error from server (BadRequest): container "espace-client-client" in pod "espace-client-client-6c8b88f795-wcrlh" is waiting to start: trying and failing to pull image
-- Juliatzin
docker
docker-registry
gitlab
kubernetes
kubernetes-secrets

1 Answer

9/23/2019

You have to update the gitlab-registry secret because this item is used to let Kubelet to pull the protected image using credentials.

Please, delete the old secret with kubectl -n yournamespace delete secret gitlab-registry and recreate it typing credentials:

kubectl -n yournamespace create secret docker-registry my-secret --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD[ --docker-email=DOCKER_EMAIL]

where: - DOCKER_REGISTRY_SERVER is the GitLab Docker registry instance - DOCKER_USER is the username of the robot account to pull images - DOCKER_PASSWORD is the password attached to the robot account

You could ignore docker-email since it's not mandatory (note the square brackets).

-- prometherion
Source: StackOverflow