How different is the Flink deployment on Kubernetes and Native Kubernetes

8/5/2020

What are the major difference b/w Native Kubernetes and Kubernetes deployments?

I'm new to Kubernetes and trying to understand how different is the Flink deployments on them. If any insight into internals is given it will be of great help.

-- ardhani
apache-flink
flink-streaming
kubernetes

1 Answer

8/5/2020

In a Kubernetes session or per-job deployment, Flink has no idea it's running on Kubernetes. In this mode, Flink behaves as it does in any standalone deployment (where there is no cluster framework available to do resource management). Kubernetes just happens to be how the infrastructure was created, but as far as Flink is concerned, it could have been bare metal. You will have to arrange for kubernetes to create the infrastructure that you will have configured Flink to expect.

In a Native Kubernetes session deployment, Flink uses its KubernetesResourceManager, which submits a description of the cluster it wants to the Kubernetes ApiServer, which creates it. As jobs come and go, and the requirements for task managers (and slots) go up and down, Flink is able to obtain and release resources from kubernetes as appropriate.

In Application Mode (blog post) (details) you end up with Flink running as a kubernetes application, which will automatically create and destroy cluster components as needed for the job(s) in one Flink application.

-- David Anderson
Source: StackOverflow