If one can produce and consume from one topic, does that indicate that every other topic in kafka is in a healthy state?

10/11/2018

If one can push and consume from one topic in a kafka cluster of 3 brokers running in kubernetes with auto_scaling disabled; can that be used to draw a conclusion that we can successfully push and consume from all the other topics in a cluster?

Total brokers in the cluster: 3
Min-InSyncReplicas: 2
No. of partitions for eahc topic: 25
-- Komal-SkyNET
apache-kafka
docker
kubernetes

1 Answer

10/11/2018

It depends. If you can produce and consume from a particular topic this is a good indication that everything is working in terms of 'communication' (successful consumption from topic_a does not mean that consumption from topic_b will also be OK). However, 'healthy state' is a quite subjective term. Kafka exposes several metrics via JMX for Brokers, Consumers and Producers. Depending on your monitoring requirement you can choose the appropriate metric and decide/judge whether everything is OK (according to your requirements).

Note: If you are using Enterprise Confluent Platform these metrics can also be observed through Confluent Control Center.

-- Giorgos Myrianthous
Source: StackOverflow