I have a small application comprising three services:
Ideally I'd like the all to be served from the same subdomain and static IP address and have been trying to configure this in Kubernetes on the Google Cloud Platform.
I've created deployments for each of the services, with the SPA exposing port 80, the API 3000 and the MQTT broker 1883/9001. I've then followed the instructions here to set up a static IP and a Service to route to the SPA, then created similar services for the API and the MQTT app. (I've initally adapted these from deployments and services generated from a docker-compose file and Kompose).
The SPA and API seem to work fine but the MQTT service does not. When I run kubetl get events I see:
Error creating load balancer (will retry): failed to ensure load balancer for service default/mqtt-broker: failed to create forwarding rule for load balancer (a5529f2a9bdaf11e8b35d42010a84005(default/mqtt-broker)): googleapi: Error 400: Invalid value for field 'resource.IPAddress': '35.190.221.113'. Specified IP address is in-use and would result in a conflict., invalid
So I'm wondering if I should be creating a single service to route to the three deployments but can't find any documentation or examples that explain how to do this for a non http service.
I guess I could put the mqtt service on a separate IP address but this seems to be hacking around the problem rather than solving it.
Thanks in advance for any advice.
I eventually found an almost identical use case to my own on this github repository.
In essence, they are creating the MQTT broker on a separate static IP and using Kubernetes API calls to expose the details to the front end, which they explain in the following comment at the top of the web.yaml file:
This needs a bit of trickery as it needs to expose the LB ip address for the MQTT server. That requires kubernetes API calls to look it up, and the ability to store it somewhere (we put it in a secret). To be secure this is done with a dedicated service account and an init container.