Have got an Apache Airflow instance placed in kubernetes cluster: webserver, scheduler and postgresql. Using custom helm charts built upon bitnami's with some changes.
Airflow is working with KubernetesExecutor. All my DAGs are PythonOperator and KubernetesPodOperator (former DockerOperator - before k8s). Xcom pushes work correctly only with PythonOperator, but with KubernetesPodOperator I'm getting errors at the end of its execution (all the dags are affected):
[2019-12-06 15:12:40,116] {logging_mixin.py:112} INFO - [2019-12-06 15:12:40,116] {pod_launcher.py:217} INFO - Running command... cat /airflow/xcom/return.json
[2019-12-06 15:12:40,201] {logging_mixin.py:112} INFO - [2019-12-06 15:12:40,201] {pod_launcher.py:224} INFO - cat: can't open '/airflow/xcom/return.json': No such file or directorySo it seems that this file is not created.
I've also tried to override post_execute method to create this file there and to json.dump the results, but it didn't help - this error still persists.
Would appreciate for any suggestions on how to resolve it.
UPDATE: I've also copy-pasted this code to my DAG https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L315, and I'm still getting this error even using apache/airflow code for unit tests.
Also have to mention that my kubernetes version is 1.11.10. Minikube 1.5.2
By default the xcom_push argument of the KubernetesPodOperator is True, which causes AirFlow to try to read /airflow/xcom/return.json from the executed containers. Just change it to False:
KubernetesPodOperator(
....
xcom_push=False
)
Changed the database (PostgreSQL) dependency and version to the newer one and got it working.