aws Ec2 Container registry (When deployed using kubernetes Image cannot be pulled due to auth credentials)

5/9/2018
  • I have created my cluster using kubeadm init and even provided the config file to define the cloud provider as aws.
  • I have given the required permission for the IAM role for ECR and i am able to pull the image using the docker pull.
  • It's just when i deploy my pod via kubernetes it cannot pull the images.

Is there something that i am missing?

Following is the description of the POD.

Events:
  Type     Reason                 Age              From                                    Message
  ----     ------                 ----             ----                                    -------
  Normal   Scheduled              4m               default-scheduler                       Successfully assigned red to ip-192-31-94-118.ec2.internal
  Normal   SuccessfulMountVolume  4m               kubelet, ip-192-31-94-118.ec2.internal  MountVolume.SetUp succeeded for volume "default-token-fv9mk"
  Normal   SandboxChanged         4m               kubelet, ip-192-31-94-118.ec2.internal  Pod sandbox changed, it will be killed and re-created.
  Normal   Pulling                3m (x3 over 4m)  kubelet, ip-192-31-94-118.ec2.internal  pulling image "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/testapp:latest"
  Warning  Failed                 3m (x3 over 4m)  kubelet, ip-192-31-94-118.ec2.internal  Failed to pull image "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/testapp:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/v2/testapp/manifests/latest: no basic auth credentials
  Warning  Failed                 3m (x3 over 4m)  kubelet, ip-192-31-94-118.ec2.internal  Error: ErrImagePull
  Normal   BackOff                2m (x7 over 4m)  kubelet, ip-192-31-94-118.ec2.internal  Back-off pulling image "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/testapp:latest"
  Warning  Failed                 2m (x7 over 4m)  kubelet, ip-192-31-94-118.ec2.internal  Error: ImagePullBackOff
-- DevScript
amazon-web-services
aws-ecr
docker
kubernetes

1 Answer

5/14/2018

When you configure Kubernetes with ECR on AWS, you can stumble in two places:

IAM Roles:

Make sure you add permissions to ALL Kubernetes nodes to pull images from ECR.

Cloud Provider: Make sure your kubelet runs with --cloud-provider=awsparameter.

In your case, you are sure all run correctly, so you missed IAM permission on some Kubernetes node.

-- Nick Rak
Source: StackOverflow