I am learning StackExchange.Redis, and Kubernetes, so I made a simple .net core app that reads a key/value from a Redis master+2slaves deployed on kubernetes. (so, everything, Redis and my app, run inside containers)
To connect to redis I use the syntax suggested in the doc:
ConnectionMultiplexer.Connect("server1:6379,server2:6379,server3:6379");However, if I monitor the 3 containers with redis-cli MONITOR, the requests are processed always from the master, the 2 slaves do nothing, so there is no load balancing.
I have tried also to connect to a Kubernetes load balancer service which exposes the 3 Redis containers endpoints, the result is that when I start the .net app the request is processed randomly by one of the 3 Redis nodes, but then always on the same node. I have to restart the .net app and it will query another node but subsequent queries go always on that node.
What is the proper way to read key/values in a load balanced way using StackExchange.Redis with a master/slave Redis setup?
Thank you
SE.Redis has a CommandFlags parameter that is optional on every command. There are some useful and relevant options here:
DemandMasterPreferMasterDemandSlavePreferSlaveThe default behaviour is PreferMaster; write operations bump that to DemandMaster, and there are a very few commands that actively prefer replicas (keyspace iteration, etc).
So: if you aren't specifying CommandFlags, then right now you're probably using the default: PreferMaster. Assuming a master exists and is reachable, then: it will use the master. And there can only be one master, so: it'll use one server.
A cheap option for today would be to add PreferSlave as a CommandFlags option on your high-volume read operations. This will push the work to the replicas if they can be resolved - or if no replicas can be found: the master. Since there can be multiple replicas, it applies a basic rotation-based load-balancing scheme, and you should start to see load on multiple replicas.
If you want to spread load over all nodes including masters and replicas... then I'll need to add new code for that. So if you want that, please log it as an issue on the github repo.
As per the docs it should automatically detect master/slaves. It may be that StackExchange.Redis is detecting that all your nodes are masters, thus just selecting one using its own tiebreaker rules.
I would also check the request logs on your redis-pods if there are some invalid commands that are being sent by StackExchange.Redis, maybe you don't have the right permissions for it to detect master/slaves.
Maybe you also have Sentinel enabled and StackExchange doesn't support sentinel
If you see something is not working you can open an issue on Github
Finally, you could also try twemproxy.