panic: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
I am trying to auto-scale with HPA with kafka metrics as per the steps given in this URL https://medium.com/google-cloud/kubernetes-hpa-autoscaling-with-kafka-metrics-88a671497f07
However, when I am deploying the yaml file given in step 3, the pod doesn't come up. The log displays the below error.
panic: kafka: client has run out of available brokers to talk to (Is your cluster reachable?) goroutine 1 [running]: main.NewExporter(0xc4200bb7e0, 0x2, 0x2, 0x100, 0x96d438, 0x0, 0x96d438, 0x0, 0x0, 0x96d438, ...) /home/travis/gopath/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:185 +0xbbc main.main() /home/travis/gopath/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:606 +0x3aa7
Similar Questions
1 Answer
Looking at the prerequisites in provided URL:
- You have a Docker running. You know the rules of this game. ;)
- You have a Kubernetes cluster (GKE) running on GCP.
- You have kubectl CLI installed and configured to your GKE cluster.
These fail to mention the presence for a kafka-cluster and the steps do not include any kafka deployments. So how exactly are you going to export metrics from kafka?
In step 3 the spec.template.spec.containers[0].command path in yaml clearly defines two brokers to use:
- "--kafka.server=my-kafka-broker-1:9092"
- "--kafka.server=my-kafka-broker-2:9092"If these are non-existent, ofcourse the kafka exporter will throw an error stating there are no brokers!