Airflow having subdags stuck when using KubernetesExecutor

1/9/2020

I have setup an airflow instance on eks and I set AIRFLOW__CORE__EXECUTOR=KubernetesExecutor. When a dag has only tasks but not a subdag then each pod is dynamically created for each tasks and runs fine but when I have a dag with multiple subdags, it does not runs the subdag and instead the first task inside the first subdag remains in the queue state.

After I set AIRFLOW__CORE__EXECUTOR=LocalExecutor and while creating the subdag I set executor=KubernetesExecutor() it works fine and tasks inside the subdags runs in separate pods. But I don't know whether I will get the same power of the KubernetesExecutor or not.

Thanks in advance.

-- user1772390
airflow
executor
kubernetes

0 Answers