Pushing Docker images from Kubernetes Pod Operator in Airflow cloud composer

2/12/2021

I am trying to push an image to GCR using KubernetesPodOperator but I keep getting this error: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication\n

My hypothesis is that the default service-account that kubernetes pod operator uses does not have the permission to push images to GCR. Here is the code that I use:

KubernetesPodOperator(
       task_id=f"push_image_to_gcr",
       name=f"push-image",
       ## push_to_gcr.py is a python script that tries to push the image to GCR using subprocess library
       cmds=["python", "push_to_gcr.py"],
       namespace="default",
       ## the image I pass in here has docker installed in it which allows me to run docker commands
       image="gcr.io/project-id/image"
)

I tried to pass in my service-account JSON file by storing the JSON under variable in Airflow UI (env_vars={'GOOGLE_APPLICATION_CREDENTIALS': '{{ var.value.my_gcp_account }}}'}), but still could not push the image.

Any help would be appreciated.

-- user9933484
airflow
google-cloud-composer
kubernetes
kubernetes-pod

0 Answers