Broker may not be available error Kafka schema registry

1/17/2019

I have defined Kafka and Kafka schema registry configuration using Kubernetes deployments and services. I used this link as a reference for the environment variables set up. However, when I try to run Kafka with registry I see that the schema registry pods crashes with an error message in the logs:

[kafka-admin-client-thread | adminclient-1] WARN
org.apache.kafka.clients.NetworkClient - [AdminClientclientId=adminclient-1] Connection to node -1
(localhost/127.0.0.1:9092) could not be established. Broker may not be
available.
    [main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.
    java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for
a node assignment.

What could be the reason of this error?

apiVersion: v1
kind: Service
metadata:
  name: kafka-service
spec:
  ports:
    - name: client
      port: 9092
  selector:
    app: kafka
    server-id: "1"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-1
spec:
  selector:
    matchLabels:
      app: kafka
      server-id: "1"
  replicas: 1
  template:
    metadata:
      labels:
        app: kafka
        server-id: "1"
    spec:
      volumes:
        - name: kafka-data
          emptyDir: {}
      containers:
        - name: server
          image: confluentinc/cp-kafka:5.1.0
          env:
            - name: KAFKA_ZOOKEEPER_CONNECT
              value: zookeeper:2181
            - name: KAFKA_ADVERTISED_LISTENERS
              value: PLAINTEXT://localhost:9092
            - name: KAFKA_BROKER_ID
              value: "2"
            - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
              value: "1"
          ports:
            - containerPort: 9092
          volumeMounts:
            - mountPath: /var/lib/kafka
              name: kafka-data
---
apiVersion: v1
kind: Service
metadata:
  name: schema-registry-service
spec:
  ports:
    - name: client
      port: 8081
  selector:
    app: kafka-schema-registry
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-schema-registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-schema-registry
  template:
    metadata:
      labels:
        app: kafka-schema-registry
    spec:
      containers:
        - name: kafka-schema-registry
          image: confluentinc/cp-schema-registry:5.1.0
          env:
            - name: SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL
              value: zookeeper:2181
            - name: SCHEMA_REGISTRY_HOST_NAME
              value: localhost
            - name: SCHEMA_REGISTRY_LISTENERS
              value: "http://0.0.0.0:8081"
            - name: SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS
              value: PLAINTEXT://localhost:9092
          ports:
            - containerPort: 8081
-- Cassie
apache-kafka
confluent-schema-registry
kubernetes

2 Answers

4/3/2020

I was getting the same kind of error and I followed cricket_007's answer and it got resolved.

Just change KAFKA_ADVERTISED_LISTENERS value from PLAINTEXT://localhost:9092 to PLAINTEXT://schema-registry-service:9092

-- Sukalpo
Source: StackOverflow

1/17/2019

You've configured Schema Registry to look for the Kafka broker at kafka:9092, but you've also configured the Kafka broker to advertise its address as localhost:9092.

I'm not familiar with Kubernetes specifically, but this article describes how to handle networking config in principle when using containers, IaaS, etc.

-- Robin Moffatt
Source: StackOverflow