Knative Kafka channel dispatcher cannot reach Kafka outside Kubernetes cluster

4/7/2019

I'm trying to setup a Kafka channel dispatcher in Knative using this documentation: https://github.com/knative/eventing/tree/master/contrib/kafka/config

Everything went great except that the channel dispatcher cannot find my Kafka broker. Error from the pod kafka-channel-dispatcher-0:

{"level":"fatal","ts":1554634078.4035408,"caller":"dispatcher/main.go:65","msg":"unable to create kafka dispatcher.","error":"unable to create kafka client: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)","stacktrace":"main.main\n\t/home/michiel/go/src/github.com/knative/eventing/contrib/kafka/cmd/dispatcher/main.go:65\nruntime.main\n\t/usr/lib/go-1.10/src/runtime/proc.go:198"}

I have a single Kafka node running in a different machine than the K8s cluster but they are in the same network. I tried reaching the Kafka broker without using Knative by running a container with some python and that was successful so I know the broker is reachable.

Here is the Kafka channel dispatcher config map:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-channel-controller-config
  namespace: knative-eventing
data:
  # Broker URL's for the provisioner. Replace this with the URL's for your kafka cluster.
  bootstrap_servers: 10.142.15.232:9092

  # Consumer mode to dispatch events from different partitions in parallel.
  # By default(multiplex), partitions are multiplexed with a single go channel.
  # `multiplex` and `partitions` are valid values.
  ## consumer_mode: partitions

What can I do to reach my broker? Does it maybe have something to do with the fact that I provide an IP-address instead of a url like in the doc example:

# Broker URL's for the provisioner
bootstrap_servers: my-cluster-kafka-bootstrap.my-kafka-namespace:9092

Update: I'm able to reach a strimzi kafka cluster like in the documentation.

-- Stanko
apache-kafka
knative
kubernetes

0 Answers