I'm using https://github.com/Yolean/kubernetes-kafka to run kubernetes on minikube I exposed an outside port and successfully used producers and consumers from outside the cluster
➜ ~ kubectl get svc --namespace kafka
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
AGE
bootstrap ClusterIP 10.108.21.84 <none> 9092/TCP
1h
broker ClusterIP None <none> 9092/TCP
1h
outside-0 NodePort 10.99.182.13 <none> 32400:32400/TCP
1h
outside-1 NodePort 10.108.10.223 <none> 32401:32401/TCP
1h
outside-2 NodePort 10.101.155.122 <none> 32402:32402/TCP
1h
pzoo ClusterIP None <none> 2888/TCP,3888/TCP
1h
zoo ClusterIP None <none> 2888/TCP,3888/TCP
1h
zookeeper ClusterIP 10.97.17.36 <none> 2181/TCP
1h
python producer
from kafka import KafkaConsumer, KafkaProducer
KAFKA_TOPIC = 'demo'
KAFKA_BROKERS = '192.168.99.100:32400' # see step 1
producer = KafkaProducer(bootstrap_servers=KAFKA_BROKERS)
messages = [b'hello kafka', b'Falanga', b'3 test messages']
for m in messages:
print(f"sending: {m}")
producer.send(KAFKA_TOPIC, m)
producer.flush()
Simple consumer
#!/usr/bin/env python
from kafka import KafkaConsumer
KAFKA_TOPIC = 'demo'
KAFKA_BROKERS = '192.168.99.100:32400' # see step 1
consumer = KafkaConsumer(KAFKA_TOPIC, bootstrap_servers=KAFKA_BROKERS)
for message in consumer:
print(f"message is of type: {type(message)}")
print(message)
print('yo')
consumer.subscribe([KAFKA_TOPIC])
How do I access the service from pods in 'default' namespace by name not using the out facing url?
You can access to the Service in the pod using the following ways.
Using the env variables,
https://kubernetes.io/docs/concepts/services-networking/service/#discovering-services
Using internal service name, i.e.> YOURSERVICENAME.NAMESPACE.svc.cluster.local
https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#services