What is the suggested volume method to use for zookeeper and broker for Apache kafka?

4/22/2019

What is the suggested volume method to use for zookeeper & broker for Apache Kafka running in the kubernetes environment?

I'm running a 3 node cluster and tried using persistent volume, but is there any recommended volume method by Apache Kafka?

-- Pooja
apache-kafka
apache-zookeeper
kubernetes
persistent-storage

1 Answer

4/22/2019

You can use helm charts to install Apache Kafka on Kubernetes (this is the recommended way by Confluent). You can find the charts here. Also, you have in that charts some default persistence sizes for Kafka. Also Confluent recommends using Shared Storage rather than Local Storage. You can find more details here and here.

The recommended storage is: 4gb for Zookeeper and for Kafka you have: r4.xlarge on AWS, n1-highmem-4 on GCP, DS5v2 on Azure and 12 core, 128GB RAM, 12 disks server on-premises. You cand find more here.

-- Dina Bogdan
Source: StackOverflow