I have written few c++ services which have the MQTT Client. Based on the message received on the MQTT topic the c++ service will take some actions like sending an MQTT message to another topic or saving the message to the database etc.
I have set up a few MQTT Brokers on Dockers and attached those MQTT Brokers to an HA Load balancer. All these MQTT Brokers also clustered.
So, if client 1 connected broker-1 ( through Load balancer ) can send message to client x connected broker -x. Due to the clustering of the MQTT Brokers.
So, How can I set the load balancer to my c++ services with HA or similar load balancers?
Update:
In the case of HTTP / REST APIs, the request will be transferred to only one web application at any point of time. But in case of MQTT, the message will be published, and If I run multiple c++ service of Same ABC then all the services will process that message. How I should make sure only one service will process the message. I want to establish High Availability for the C++ service
This is not possible under MQTT 3.x. The reason being that prior to MQTT 5, every message is sent to every subscriber to that topic making it very difficult to load balance correctly. Subscribers would need receive everything then discard decide for themselves to discard some messages, leaving them for other subscribers. It's one of the limitations of MQTT 3.x.
There are those who have worked around this by connecting their MQTT broker into an Apache Kafka cluster, routing all messages from MQTT to Kafka and then attaching their subscribers (like your c++ services) to Kafka instead of MQTT. Kafka supports the type of load balancing you are asking for.
This may be about to change with MQTT 5.0. There are still a lot of clients and brokers which don't support this. However if both your client and broker support MQTT version 5 then there is a new 1 concept of "Shared Subscriptions":
- Shared Subscriptions – If the message rate on a subscription is high, shared subscriptions can be used to load balance the messages across a number of receiving clients
You haven't stated your client library. But your first steps should be:
1 New to MQTT, Kafka already has it.