Serving MQTT over WebSocket in Kubernates GCP environment

10/18/2019

I'm currently serving MQTT messages over WebSocket to js clients. I use RabbitMQ to write messages on queue from a java backend and have them routed to the clients/frontend apps. I deployed everything on a Kubernetes cluster on Google Cloud Platform and everything works just fine as long as I publish the RabbitMQ pod with a Kubernetes Load Balancer directly to the internet.

    apiVersion: v1
kind: Service
metadata:
  labels:
    app: rabbitmq
  name: rabbitmq
spec:
  type: LoadBalancer
  ports:
  - name: http-manager
    nodePort: 30019
    port: 80
    protocol: TCP
    targetPort: 15672
  - name: mqtt-broker
    nodePort: 31571
    port: 1883
    protocol: TCP
    targetPort: 1883
  - name: ws-service
    nodePort: 32048
    port: 15675
    protocol: TCP
    targetPort: 15675
  selector:
    app: rabbitmq

I try to replace the Kubernetes Load balancer with a Node port service and expose it through an Ingress and a GCP Balancer but the health probe fails and never recovers.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: rabbitmq
  name: rabbitmq-internal
spec:
  ports:
  - name: ws-port
    port: 15675
    protocol: TCP
    targetPort: 15675
  - name: mamanger-port
    port: 15672
    protocol: TCP
    targetPort: 15672
  selector:
    app: rabbitmq
  sessionAffinity: None
  type: NodePort


apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: basictest
  namespace: default
spec:
  rules:
  - host: mqtt-host.dom.cloud
    http:
      paths:
      - backend:
          serviceName: rabbitmq-internal
          servicePort: 15675
        path: /ws/*
      - backend:
          serviceName: rabbitmq-internal
          servicePort: 15672
        path: /*

The probe is HTTP so I tried to assign a custom TCP probe and even to trick GCP switching with a probe that points to another HTTP port on the same pod, with no success.

I need to user GCP Balancer to have a unified frontend to assign an SSL Certificate for both HTTPS and WSS protocols.

-- Sammyrulez
google-cloud-platform
google-kubernetes-engine
kubernetes
kubernetes-ingress
websocket

0 Answers