Kubernetes: Load balancing the consumers of a RabbitMQ exchange

11/1/2018

First off, I am almost certain this is already a solved problem. However, I cannot for the life of me find the right combination of search terms to find it anywhere.

Picture a simple RabbitMQ-exchange-based communication on a Kubernetes Cluster using microservices. There is a central RabbitMQ deployment through which communications are broadcast (using a topic exchange) and then consumed by various microservices. The issue I have is that each pod of each microservice connects directly to the exchange, which leads to the problem that a broadcast message may be processed several times if a microservice has scaled up to use several pods.

For a minimal example, imagine two deployments: A RabbitMQ deployment with one pod, and a "Log" deployment with several pods, each running an instance of the same "Log" microservice. All that the Log microservice does is consume all messages from the RabbitMQ exchange and saves them to a database. Since each instance of the microservice independently connects to the exchange (through the amqp protocol), each of them will independently consume each message and write it to the database. So if there are five pods, there would be five entries in the database when I only want one.

So my question is: In Kubernetes, how do I best load-balance between dynamically scaled consumers of a topic exchange, the number of which I do not know at runtime, to make sure that a message from an exchange only ever reaches one of them?

-- Hoshi
kubernetes
load-balancing
rabbitmq

0 Answers