I am getting started with Airflow and trying to use the KubernetesPodOperator, but I am having trouble with downloading images from private registries. I did some research but I couldn't find an answer to my problem.
Putting it simply: can I use private images from DockerHub with the KubernetesPodOperator?
Yes, you can. The KubernetesPodOperator accepts an imagePullSecret since version 1.10.1
:param image_pull_secrets: Any image pull secrets to be given to the pod.
If more than one secret is required, provide a
comma separated list: secret_a,secret_b
:type image_pull_secrets: strReferences: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ https://github.com/apache/airflow/blob/master/airflow/contrib/operators/kubernetes_pod_operator.py
It looks like pod_generator.PodGenerator accepts some kind of object kube_config that knows about imagePullSecrets, but unfortunately KubernetesPodOperator doesn't provide any such kube_config to PodGenerator
As best I can tell, it's just an edge case that slipped through the cracks, although it looks like there is a Jira for that which matches up with a corresponding GitHub PR, but it isn't clear from looking at the changed files that it will 100% solve the problem you are describing. Perhaps weigh in on either the PR, or the Jira, or maybe even both, to ensure it is addressed.