K
Q

Create a redis sentinel cluster in Kubernetes, the redis master can not get the slave origin IP

5/10/2016

I use the K8s example the create a Reliable, Scalable Redis on Kubernetes. https://github.com/kubernetes/kubernetes/tree/master/examples/redis

The Redis master needs to get slave the origin IP and register to the sentinel. Sentinel will use to switch master when the master failed.

But My docker starts without the flag iptables=false(It will create an iptables like SNAT), so the redis slave source IP will be SNAT, and the master will get the flannel0's IP like 172.16.103.0.

I also add the flannel network to container connection between nodes.

So the question is "Is any way to get the source IP without removing the iptables flag, or another way to create a Reliable redis cluster."

-- zhulinhong
kubernetes
redis-sentinel

Similar Questions

Is it appropriate to run one kubernetes cluster for two datacenters
Run kubernetes from source and configure cloud provider
"403 Insufficient Permission" while creating a Kubernetes Cluster on Google Cloud instance
how to autoscale node kubernetes

2 Answers

5/31/2016

We managed to work around this by setting hostNetworking: true in our redis pods. This means the redis servers don't get container IPs (172..) and is definitely a work around but fixes the issue.

-- Andy Smith
Source: StackOverflow
5/24/2016

EDIT: I have figured out the magic combination of flags to get this working.

1) run stop docker & the bootstrap docker processes

systemctl stop docker && pkill -f "docker-bootstrap" && sleep 10

2) Delete iptables rules:

iptables -F && iptables -t nat -F

3) run your docker-daemon with the ip-masq=false option
4) run your flanneld with the ip-masq=true option

For me this makes all three pod->pod, pod->ext, & ext->service->pod work perfectly.

Hope this works for the rest of you!

(Preserving initial response which described the symptoms)

I am having the same problem with the guide. The slave replicates just fine but the master can't properly check the slave status because the connection is coming from the dot zero ip.

Proto Recv-Q Send-Q Local Address           Foreign Address         State

Example slave->master connection from the master perspective (shows wrong IP of slave):

tcp   0      0 redis-master:6379       10.1.37.0:51674         ESTABLISHED

Example slave->master connection from slave perspective (shows right ip of master):

tcp   0      0 redis-03fdy:51674       10.1.90.2:6379          ESTABLISHED

Here are the redis server commands from ps.

master:

root   12  0.1  0.2  26876  2352 ?   Sl   10:42   0:02 redis-server 10.1.90.2:6379

slave:

root   18  0.1  0.2  26876  2300 ?   Sl   10:48   0:00 redis-server 10.1.37.3:6379
-- ssjcory
Source: StackOverflow