Is there something that i am missing?
Following is the description of the POD.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 4m default-scheduler Successfully assigned red to ip-192-31-94-118.ec2.internal
Normal SuccessfulMountVolume 4m kubelet, ip-192-31-94-118.ec2.internal MountVolume.SetUp succeeded for volume "default-token-fv9mk"
Normal SandboxChanged 4m kubelet, ip-192-31-94-118.ec2.internal Pod sandbox changed, it will be killed and re-created.
Normal Pulling 3m (x3 over 4m) kubelet, ip-192-31-94-118.ec2.internal pulling image "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/testapp:latest"
Warning Failed 3m (x3 over 4m) kubelet, ip-192-31-94-118.ec2.internal Failed to pull image "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/testapp:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/v2/testapp/manifests/latest: no basic auth credentials
Warning Failed 3m (x3 over 4m) kubelet, ip-192-31-94-118.ec2.internal Error: ErrImagePull
Normal BackOff 2m (x7 over 4m) kubelet, ip-192-31-94-118.ec2.internal Back-off pulling image "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/testapp:latest"
Warning Failed 2m (x7 over 4m) kubelet, ip-192-31-94-118.ec2.internal Error: ImagePullBackOff
When you configure Kubernetes with ECR on AWS, you can stumble in two places:
IAM Roles:
Make sure you add permissions to ALL Kubernetes nodes to pull images from ECR.
Cloud Provider: Make sure your kubelet runs with --cloud-provider=aws
parameter.
In your case, you are sure all run correctly, so you missed IAM permission on some Kubernetes node.