I am trying to configure a Kafka cluster behind Traefik but my producers and client (that are outside kubernetes) don't connect to the bootstrap-servers. They keep saying:
"no resolvable boostrap servers in the given url"
Actually here is the Traefik ingress:
{
"apiVersion": "extensions/v1beta1",
"kind": "Ingress",
"metadata": {
"name": "nppl-ingress",
"annotations": {
"kubernetes.io/ingress.class": "traefik",
"traefik.frontend.rule.type": "PathPrefixStrip"
}
},
"spec": {
"rules": [
{
"host": "" ,
"http": {
"paths": [
{
"path": "/zuul-gateway",
"backend": {
"serviceName": "zuul-gateway",
"servicePort": "zuul-port"
}
},
{
"path": "/kafka",
"backend": {
"serviceName": "kafka-broker",
"servicePort": "kafka-port"
}
[..]
}
What I give to the kafka consumers/producers is the public IP of Traefik. Here is the flow: [Kafka producers/consumers] -> Traefik(exposed as Load Balancer) -> [Kafka-Cluster]
Is there any solution? Otherwise I was thinking to add a kafka-rest proxy (https://docs.confluent.io/current/kafka-rest/docs/index.html) between Traefik and the kafka brokers but I think isn't the ideal solution.
I did. You can refer to it, in kubernetes ,deployment kafka.yaml
env:
- name: KAFKA_BROKER_ID
value: "1"
- name: KAFKA_CREATE_TOPICS
value: "test:1:1"
- name: KAFKA_ZOOKEEPER_CONNECT
value: "zookeeper:2181"
- name: KAFKA_ADVERTISED_LISTENERS
value: "INSIDE://:9092,OUTSIDE://kafka-com:30322"
- name: KAFKA_LISTENERS
value: "INSIDE://:9092,OUTSIDE://:30322"
- name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
value: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
- name: KAFKA_INTER_BROKER_LISTENER_NAME
value: "INSIDE" kafka service,the external service invocation address, or traefik proxy address
---
kind: Service
apiVersion: v1
metadata:
name: kafka-com
namespace: dev
labels:
k8s-app: kafka
spec:
selector:
k8s-app: kafka
ports:
- port: 9092
name: innerport
targetPort: 9092
protocol: TCP
- port: 30322
name: outport
targetPort: 30322
protocol: TCP
nodePort: 30322
type: NodePortEnsure that Kafka external port and nodePort port are consistent,Other services call kafka-com:30322, my blog write this config_kafka_in_kubernetes, hope to help U !