I am having trouble subscribing and publishing to my Mosquitto server running on my Kubernetes cluster.
I know my service and pod are working. I can subscribe and publish from inside the cluster using my mosquitto service as host.
I also have a working HTTPS gateway so I can access https://mosquittourl.com.br from outside and I can see it reaches my mosquitto container on port 1883. Container logs:
New connection from 127.0.0.1 on port 1883.
Client <unknown> disconnected due to protocol error.
I guess this is the expected behavior. Because I am accessing it via https instead of mqtt.
Now, I can't connect to the container using:
$ mosquitto_pub --url mqtt://mosquittourl.com.br:<port>/test
How do I correctly set up my Istio resources so I can subscribe and publish to my mosquitto server? Is there a way to configure Istio to accept mqtt requests and redirect them to my mosquitto service?
I was recently able to get Istio setup to route TLS encrypted MQTT connections to
Mosquitto running in a container on Kubernetes. You should use TLS as the protocol
for the port in the Istio Gateway. Eventually Istio might support MQTT as an option like it does with gRPC and Mongo, but for now you need to use TCP for unencrypted MQTT traffic, and TLS for secure MQTT connections.
There are few things you need to make sure to configure correctly to get this working.
Istio Operator Profile yaml file
For your IstioOperator
that configures the main load balancer for Istio, you want to make sure to setup the correct ports. For secure MQTT traffic, that is 8883 (technically you can use any port but 8883 is the default).
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
accessLogFile: /dev/stdout
components:
egressGateways:
- name: istio-egressgateway
enabled: true
k8s:
resources:
requests:
cpu: 10m
memory: 40Mi
ingressGateways:
- name: istio-ingressgateway
enabled: true
k8s:
resources:
requests:
cpu: 10m
memory: 40Mi
service:
ports:
- port: 15021
targetPort: 15021
name: status-port
- port: 31400
targetPort: 31400
name: tcp
# This is the port where sni routing happens
- port: 15443
targetPort: 15443
name: tls
- port: 8883
targetPort: 8883
name: mqtt-secure
pilot:
k8s:
env:
- name: PILOT_TRACE_SAMPLING
value: "100"
resources:
requests:
cpu: 10m
memory: 100Mi
values:
global:
proxy:
resources:
requests:
cpu: 10m
memory: 40Mi
pilot:
autoscaleEnabled: false
gateways:
istio-egressgateway:
autoscaleEnabled: false
istio-ingressgateway:
autoscaleEnabled: false
runAsRoot: true
The important part here is to make sure you specify the name of the SSL certificate you already have in your Kubernetes secrets, and to use the same port you did in the IstioOperator, and make sure the protocol is TLS. The tls:
block tells the gateway to do TLS termination using the specified SSL certificate.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: my-mqtt-ssl-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 8883
name: mqtt-secure
protocol: TLS
hosts:
- "*"
tls:
credentialName: cert-my.certname.com
mode: SIMPLE
privateKey: sds
serverCertificate: sds
The final step is to setup an Istio Virtual Service that takes the traffic from the Istio Gateway and routes it to the correct pod. Update your pod name in the destination. The other aspect is to match on port 8883, and reroute the traffic to mosquitto on port 1883 since the traffic has been TLS terminated in the Istio Gateway.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: my-mqtt-vs
spec:
hosts:
- "*"
gateways:
- my-mqtt-ssl-gateway
tcp:
- match:
- port: 8883
route:
- destination:
host: my-mqtt-broker-pod-name
port:
number: 1883