I have a running Kafka-zookeeper cluster using Strimzi cluster-operator. I have followed the KB http://strimzi.io/docs/master/ for this. The metrics for kafka and zookeeper are available from the port 9404. I have to configure this cluster setup to Prometheus for monitoring. The Strimzi itself has a KB for configuring Prometheus too(http://strimzi.io/docs/master/#assembly-metrics-deployment-configuration-kafka). But my issue is, I can't configure the cluster according to their KB, Because in my Prometheus setup I have to configure service-monitor for discovering the applications,and I need to specify the metrics available port in its service. But I could not found any solution for this. Requesting help...
the current implementation already sets the Prometheus annotations on the Kafka and Zookeeper services (they are named my-cluster-kafka-bootstrap and my-cluster-zookeeper-client). These annotations are:
prometheus.io/path /metrics prometheus.io/port 9404 prometheus.io/scrape true
But for having them, you need to enable metrics. It's possible by setting the metrics field in the Kafka resource for Kafka and Zookeeper cluster. You can find more info here: http://strimzi.io/docs/master/#assembly-metrics-deployment-configuration-kafka.
Furthermore, the repo provides you an example of Kafka cluster with metrics enabled. It's here: https://github.com/strimzi/strimzi-kafka-operator/blob/master/metrics/examples/kafka/kafka-metrics.yaml.