I am struggling to setup Kafka Connect on Kubernetes (DockerEE) in distributed mode.<br/> Currently I have a cluster of three workers on three corresponding k8s-pods.<br/> The problem that Im facing is that my workers are having a hard time communicating to each other (at least that is what I belive).<br/>
When I try to start a connector I get:
{"error_code":409,"message":"Cannot complete request because of a conflicting operation (e.g. worker rebalance)"}in response. The puzzling thing is that I dont always get this error response. Sometimes it work and the connector is started as expected.
From what Ive read it probably comes down to what I have configured as CONNECT_REST_ADVERTISED_HOST_NAME.
What is the correct value to put as advertised address when running in k8s?
BR
After some googling I found a way to dynamically assign the pod-ip as CONNECT_REST_ADVERTISED_HOST_NAME.
- name: CONNECT_REST_ADVERTISED_HOST_NAME
valueFrom:
fieldRef:
fieldPath: status.podIPWith this the problems seem to have vanished.