i saw that some people use Kubernetes co-exist with slurm, I was just curious as to why you need kubernetes with slurm? what is the main difference between kubernetes and slurm?
Slurm is open source job scheduling system for large and small Linux clusters. It is mainly used as Workload Manager/Job scheduler. Mostly used in HPC (High Performance Computing) and sometimes in BigData.
Kubernetes is an orchestration system for Docker containers using the concepts of ”labels” and ”pods” to group containers into logical units. It was mainly created to run micro-services and AFAIK currently Kubernetes is not supporting Slurm.
Slumr as Job scheduler have more scheduling options than Kubernetes, but K8s is container orchestration system not only Job scheduler. For example Kubernetes is supporting Array jobs and Slurm supports Parallel and array jobs. If you want to dive in to scheduling check this article.
As I mentioned before, Kubernetes is more focused on container orchestration and Slumr is focused on Job/Workload scheduling. Only thing comes to my mind is that someone needed very personal-customized cluster using WLM-Operator + K8s + Slurm + Singularity to execute HPC/BigData jobs.
Usually Slurm Workload Manager is used by many of the world's supercomputers to optimize locality of task assignments on parallel computers.