K
Q

Kubernetes and Kafka (Advantages, Disadvantages, and main differences)

11/1/2019

I am interested in using Kubernetes to run Kafka, Zookeeper, and other ecosystem.

I want to know what is the big difference between using Kubernetes and not using Kubernetes to run Kafka. What are the things that I should keep in mind when running a Kafka in a Kubernetes environment?

Another question is what are the advantages and disadvantages of using Kubernetes to run Kafka?

Any tips or advice will be beneficial to me. Thank you.

-- Jin Lee
apache-kafka
kubernetes

Similar Questions

Where are the Kubernetes kubelet logs located for the AWS EKS Clusters?
Getting notification on file modification within a container in kubernetes pod
Is it possible to abstract boilerplate YAML from helm subchart templates?
How does the GKE metadata server work in Workload Identity

1 Answer

11/12/2019

My question was too broad. Yet anyone who wants to deploy Kafka with Kubernetes will have similar questions beforehand. I was given some good links for tips and summarized a few important points for others here. As my expertise in Kafka and Kubernetes grows in the future, I will update my answer.

Advantages of Using Kubernetes with Kafka :

  • Easy to deploy and manage a new cluster

  • Easy to manage Kafka and to scale up

  • Easy to perform configuration changes, upgrades, and restarts

Biggest challenges with Kubernetes and Kafka :

  • Configuring storage and network (Kafka is a stateful service. But if you can get support from skilled storage and network teams, Kubernetes, overall, is the easiest way to run Kafka.)

Useful links given to me by Eduardo Baitello :

  1. Apache Kafka on Kubernetes – Could You? Should You?

  2. Recommendations for Deploying Apache Kafka® on Kubernetes.

-- Jin Lee
Source: StackOverflow