My Argo setup is on GKE.

What I observed is after a successful completion of a workflow, it's getting auto deleted. However, if a workflow fails, the workflow and all the node pods remain in the cluster.

I could not find any documentations around this. I'll be extremely grateful if you could share some pointers around this.

Note, this is happening only on GKE cluster. Running it on minikube doesn't reproduce this behaviour.