I need to setup a RabbitMQ cluster with queue mirroring enabled on all queues in Kubernetes. The RabbitMQ plugin for kubernetes peer discovery only provides a clustering mechanism based on peer discovery , as the plugin name indicates. But how do I enable queue mirroring and achieve HA , so that if pods a restarted for any reason or if I need to scale the Rabbitmq nodes , I can do it without any loss of messages.
You can enable RabbitMQ mirroring using Kubernetes lifecycle.
The following entry is how I did it. It's not perfect on openshift the first pod of the statefulset restart once at launch.
lifecycle:
postStart:
exec:
command:
- "/bin/bash"
- "-c"
- "for i in $(seq 15 -1 1); do sleep 2; done; until rabbitmqctl status; do sleep 2; done; rabbitmqctl set_policy ha \".\" '{\"ha-mode\":\"exactly\", \"ha-params\":2, \"ha-sync-mode\":\"automatic\"}'"The first loop is to wait for node to configure itself or it would interact with a temporary configuration. The second is the wait the start of the RabbitMQ.
This configuration will enable duplicating all queue with two replicas.
Also keep in mind that if the postStart failed the container will failed too. Also using a postStart will delay the readiness of the pod till the postStart command return.
Add a definitions.json file into your ConfigMap and ensure that your pods mount the file (in /etc/rabbitmq). In that file, specify all exchanges/queues, and have a policy defined for mirroring that would be applied to those exchanges/queues.
It may be easier to manually set this up and export the definitions file from a running RabbitMQ node.
This way - your cluster is all configured when started.