I'm new to nextflow. We would like to build our workflow using nextflow and have nextflow deploy the workflow to a large mulit-institution Kubernetes cluster that we use.
In this cluster we don't have admin permission, we have a namespace we work in. Also, pods in our cluster have limited resources, but jobs have unlimited resources.
In looking at the documentation for nextflow + kubernetes and it says that the workflow runs under a Kubernetes pod, which raises red flags for me because of the limitation on pods in our cluster.
Is there a way to execute nextflow workflows as a kubernetes jobs instead of a pod? What are my options in this area?
Based on a conversation on https://gitter.im/nextflow-io/nextflow, Nextflow cannot run a Job, it only supports spawning Pods. One pod is spawned for workflow control, and that Pod spawns pods for the individual tasks.