We have an application with a dependency on RabbitMQ which is deployed inside the kubernetes cluster as a standard statefulset / service style deployment (using helm stable/rabbitmq-ha). We then in a separate namespace have an application deployed in a similar style deployment / service layout but with istio enabled (including sidecar injection).
As soon as we turn on istio sidecar injection the deployment can no longer connect to the kubernetes service.
I've tried putting in an istio ServiceEntry in with the FQDN of each RabbitMQ pod but that seemingly hasn't made a difference.
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: rabbitmq
spec:
hosts:
- "rabbitmq-rabbitmq-ha-0.rabbitmq-rabbitmq-ha-discovery.rabbitmq.svc.cluster.local"
- "rabbitmq-rabbitmq-ha-1.rabbitmq-rabbitmq-ha-discovery.rabbitmq.svc.cluster.local"
- "rabbitmq-rabbitmq-ha-2.rabbitmq-rabbitmq-ha-discovery.rabbitmq.svc.cluster.local"
ports:
- number: 4369
name: epmd
protocol: TCP
- number: 5672
name: amqp
protocol: TCP
- number: 15672
name: http
protocol: HTTP
- number: 25672
name: inter-node
protocol: TCP
...I can see traffic makes it to the RabbitMQ pod but with Istio turned on it throws the following error:
2019-06-19 09:40:39.538 [info] <0.32110.48> accepting AMQP connection <0.32110.48> (10.233.122.234:47530 -> 10.233.122.85:5672)
2019-06-19 09:40:39.538 [error] <0.32110.48> closing AMQP connection <0.32110.48> (10.233.122.234:47530 -> 10.233.122.85:5672):
{bad_header,<<22,3,1,0,222,1,0,0>>}The only thought I do have at the moment is that RabbitMQ does resolve the Kubernetes DNS into each specific pod using a plugin (rabbitmq_peer_discovery_k8s), could that be interferring?
Just to be clear I have no interest in enabling Istio mTLS in between the application and RabbitMQ just to have fully functional basic connectivity.
We're using: