sql_alchemy_pool_size for kubernetes executor on airflow

5/21/2020
airflow version = 1.10.3
executor = KubernetesExecutor
airflow metadataDB = postgresDB

Does any of the airflow pool parameters(sql_alchemy_pool_size, sql_alchemy_max_overflow, sql_alchemy_pool_recycle) have any significance here with my above setup ? (Does each of the airflow kubernetes task communicate directly with the metadata DB ?). I have added that in the [core] section of my airflow.cfg but not seeing any changes in the behaviour.

We are seeing a large number of connections(40 to 70) and High CPU usage with the postgres DB?

Any solution here is greatly appreciated .

-- hopeIsTheonlyWeapon
airflow
airflow-scheduler
kubernetesexecutor

0 Answers