We have an application which uses a service-oriented architecture. It uses Apache ActiveMQ as a message bus. We have server applications installed on VMs in two geographically separated data centers. Furthermore, client applications installed on remote devices communicate with the server applications via the message bus. Next to that, there are some web applications which allow operators to interact with the server applications or control actions on the remote devices. The web applications use a combination of websockets and HTTP to communicate with the servers. All of the connections are of course encrypted.
We are thinking of migrating the server applications to containers hosted in Kubernetes or Swarm. This would help us to simplify deployment. For the most part, we can find useful information on various websites and documentation pages. However, for ActiveMQ we still find it difficult to come up with the correct setup.
Our current setup uses a network of brokers so ensure high availability and scalability. The scalability is (currently) not really necessary. A single broker could easily handle the number of messages flowing through the system. It is mainly because of the availability requirements that we have multiple brokers (one per VM).
What would be a good way to meet these requirements when running ActiveMQ (or possible Apache Artemis) inside Kubernetes? We could run a single POD and let Kubernetes take care of the HA features. But how would clients know that the ActiveMQ-POD died and Kubernetes has started a new replica, possibly in another data center?
We could of course just deploy multiple PODs which we configure as a network of brokers, similar to what we have now. But then we probably also need to generate certificates per POD to get the encryption between client and server working.
Ideally we'd only have one certificate which is connected to the single DNS name behind which our entire set of applications is running. The certificate would be configured in HAProxy or something like that. Internally we would then only use a self-signed certificate which would be part of the deliverables. In that way, we can deploy the same application to each customer-sight in exactly the same with the only variable being the one SSL certificate to encrypt traffic from outside towards the ingress-point.