GKE Containerization of Kafka Cluster

3/20/2020

I'm trying to containerize a Kafka cluster on Google Kubernetes Engine following this reference.

https://dzone.com/articles/ultimate-guide-to-installing-kafka-docker-on-kuber

While I see the Kafka-broker and Zookeepar pods up and running, write operation on any topic is failing with below error.

bash-4.4# kafka-console-producer.sh --topic test1 --broker-list PLAINTEXT://34.70.196.179:9092
>hello
[2020-03-20 11:55:24,806] WARN [Producer clientId=console-producer] Error while fetching metadata 
with correlation id 34 : {test1=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

I described the topics and found that leaders weren't getting elected.

bash-4.4# kafka-topics.sh --describe --zookeeper zoo1:2181 --topic test
Topic: test     PartitionCount: 2       ReplicationFactor: 1    Configs:
Topic: test     Partition: 0    Leader: 0       Replicas: 0     Isr: 0
Topic: test     Partition: 1    Leader: 0       Replicas: 0     Isr: 0
bash-4.4# kafka-topics.sh --describe --zookeeper zoo1:2181 --topic admintome-test
Topic: admintome-test   PartitionCount: 1       ReplicationFactor: 1    Configs:
Topic: admintome-test   Partition: 0    Leader: 0       Replicas: 0     Isr: 0

Any suggestions? Thanks in advance.

-- Balajee Venkatesh
apache-kafka
google-kubernetes-engine

0 Answers