I'm pretty new to kafka, docker and kubernetes. I was trying to run single node or multi-node kafka cluster within minikube and producing/consuming the messages to/from kafka broker using .net console application.
Can anyone direct/point me to any working solution of .net application? using which I can create a simple POC which would be able to send/receive messages to/from kafka brokers?
I've spent too much time on implementing this using solsson/kafka-persistent and confluent simple producer/consumer code, but I'm only able to access topic with in the minikube cluster, outside of cluster kafka brokers are not visible. I found another solution for the same Yolean/kubernetes-kafka but since I'm unfamiliar with kubernetes/networking and yaml I'm not able to understand properly and unable to achieve a working simple console application which can produce/consume messages to/from kafka topic from outside of kubernetes.
It would be very helpful if someone can help me with some working solution or direct me any step by step implementation of kafka messaging service using .net client.
Thanks in advance!
To expose Kafka outside of minikube (Kubernetes) is a very complicated issue, because you will always get unrouted IP addresses (internal adresses which can be changed any time) as an answer from Kafka broker. So the best way is to use .net client inside the Kubernetes cluster near Kafka. Then you will have Kafka and .net client, for example Confluent's in the same network. I can advise you to read this article, it is related to GKE, but anyway you can use it as an example, and run the same configuration in minikube.