Cloud Composer creates many Kubernetes workloads, should they be deleted?

9/18/2019

I'm using Airflow's KubernetesPodOperator on Google Cloud Composer and each time it is called a new workload appears in Google Kubernetes Engine. Once the pod scales down, the workload record is still visible in GKE. Over time, this has grown to be a significant list of workloads.

Is there any implication of having all these historical workloads?

Is this something I should be managing -- deleting unused workloads?

-- David Beaudway
airflow
google-kubernetes-engine

1 Answer

11/21/2019

You will not be charged by non running workloads, so there is no need to delete them.

Keeping a record of all the workloads that have been deployed can be useful for troubleshooting, but feel free to delete any of them if you wish.

-- Daniel Duato
Source: StackOverflow