Kubernetes Endpoint created for Kafka but not reflecting in POD

8/19/2019

In Kubernetes cluster I have created Endpoint pointing to Kafka cluster. Endpoint created successfully.

Name - kafka
Endpoint - X.X.X.X:9092

In my Spring Boot application's deployment yaml I have kept environment variable BROKER_IP. For this environment variable I have pointed:

env:
    - name: BROKER_IP
      value: kafka

The POD is in Error state. In my bootstrap-server I am getting kafka and not the actual Endpoint that was created. Any thoughts?

UPDATE - Just tried kafka:9092 and it worked. So wondering does the ENDPOINT maps to IP only and not the Port? Is my understanding correct??

-- Arpan Sharma
apache-kafka
kubernetes
kubernetes-service
spring-boot
spring-kafka

1 Answer

8/19/2019

Is it possible that you forgot to create the Service object matching the Endpoints? Because you are providing the ip-port pairs yourself the Service would need to be selectorless.

This works for me:

kind: Endpoints
apiVersion: v1
metadata:
  name: kafka
subsets:
  - addresses: [{ip: "1.2.3.4"}]
    ports: [{port: 9092}]
---
kind: Service
apiVersion: v1
metadata:
  name: kafka
spec:
  ports: [{port: 9092}]

Testing it:

$ kubectl run kafka-dns-test --image=busybox --attach --rm --restart=Never -- nslookup kafka
If you don't see a command prompt, try pressing enter.
Server:         10.96.0.10
Address:        10.96.0.10:53

Name:   kafka.default.svc.cluster.local
Address: 10.96.220.40

Successful lookup, ignore extra *** Can't find xxx: No answer messages

Also, because there is a Service object you get some environment variables in your Pods (without having to declare them):

KAFKA_PORT='tcp://10.96.220.40:9092'
KAFKA_PORT_9092_TCP='tcp://10.96.220.40:9092'
KAFKA_PORT_9092_TCP_ADDR='10.96.220.40'
KAFKA_PORT_9092_TCP_PORT='9092'
KAFKA_PORT_9092_TCP_PROTO='tcp'
KAFKA_SERVICE_HOST='10.96.220.40'
KAFKA_SERVICE_PORT='9092'

But the most flexible way to use a Service is still to use the dns name (kafka in this case).

-- Janos Lenart
Source: StackOverflow