Why Flink Job Cluster on Kubernetes?

7/11/2019

I found the designs of Flink session cluster and job cluster on Kubernetes are quite different, session cluster is clean which consists of 2 deployments and 1 service, but job cluster looks complicated, it requires building an image with the job, then take several steps to run.

My question is that, why don't design job cluster as a wrapper of a session cluster + a job, which means it automatically: 1) creates a session cluster; 2) submits the job to the cluster; 3) deletes the cluster after the job is done. Users only needs to specify the cluster spec and the job spec. What is the problem with this wrapper design? And what is the benefit of the current job cluster design?

-- Dagang
apache-flink
kubernetes

0 Answers