I have followed these links
to build a Redis sharded cluster on AKS, But they are connected by using Pod IP's but I need to connect to that cluster using "Python" to feed the data into it, Since they are connected internally using Pod IPs I am not able to connect using Python. Alternatively instead of 6 replicas of one statefulset, I have created 6 different statefulsets with 6 services and all exposed externally as "Load Balancer", But this command they used to make a cluster
kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 \
$(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
I don’t know how to edit it to use loadbalancer IP instead of Pod IP, so with this 6 external IP's I am not able to create a cluster. I need a Redis sharded cluster on Azure Kubernetes Services which can be accessible externally from Python library "redis-py-cluster"
Thanks in advance
theory
It is not very clear from your question as there is no answer formy comment under the initial post, but it looks like you have been following this guide: Setup Persistence Redis Cluster in Kubernetes.
Lets decompose the command you are referring to:
kubectl exec -it redis-cluster-0 -- redis-trib create --replicas 1 $(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
the kubectl
syntax is:
kubectl exec (POD | TYPE/NAME) [-c CONTAINER] [flags] -- COMMAND [args...] [options]
That is why: kubectl exec -it redis-cluster-0
stands for "Execute a command in a container" with "Pass stdin to the container" and "Stdin is a TTY" options on redis-cluster-0
pod.
The second part is redis-trib create --replicas 1 <IPs>
and
$(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
merely lists IPs of pods in your StatefulSet/s created with redis-cluster-deployment.yaml .
Taking into consideration that the app=redis-cluster
label we are using in that command is taken from thespec/template/metadata/labels
you can adjust the command the way you like or just list the IPs manually insted of getting them with kubectl get pods -l
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
...
spec:
serviceName: redis-cluster
replicas: 6
template:
metadata:
labels:
app: redis-cluster # This is the label we are using in that command
my answer:
You can either use the IPs or adjust labels you are using in kubectl get pods -l ...
part of the command. Can't provide precise answer as I didn't receive steps to reproduce.
my attempt:
I'm not sure how exactly you've been creating your statefulsets; however, you still can create a redis cluster on them.
In my case I have used the following YAMLs: redis-cluster-deployment-1.yaml and redis-cluster-deployment-2.yaml .
$ kubectl create -f redis-cluster-deployment-1.yaml
statefulset.apps/redis-cluster-set-1 created
$ kubectl create -f redis-cluster-deployment-2.yaml
statefulset.apps/redis-cluster-set-2 created
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
redis-cluster-set-1-0 1/1 Running 0 52m 10.12.0.22 gke-6v3n
redis-cluster-set-1-1 1/1 Running 0 51m 10.12.1.17 gke-m7z8
redis-cluster-set-1-2 1/1 Running 0 50m 10.12.1.18 gke-m7z8
redis-cluster-set-2-0 1/1 Running 0 51m 10.12.0.23 gke-6v3n
redis-cluster-set-2-1 1/1 Running 0 50m 10.12.1.19 gke-m7z8
redis-cluster-set-2-2 1/1 Running 0 14m 10.12.0.24 gke-6v3n
$ kubectl exec -it redis-cluster-set-1-0 -- redis-trib create --replicas 1 $(kubectl get pods -l app=redis-cluster-set-app -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
10.12.0.22:6379
10.12.1.17:6379
10.12.1.18:6379
Adding replica 10.12.1.19:6379 to 10.12.0.22:6379
Adding replica 10.12.0.24:6379 to 10.12.1.17:6379
Adding replica 10.12.0.23:6379 to 10.12.1.18:6379
....
the important part here is to specify correct pod name
and app=
label .
In my case , the $(kubectl get pods -l app=redis-cluster-set-app -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
command results in a following list of IPs (you can compare them with the output from above.
$ kubectl get pods -l app=redis-cluster-set-app -o jsonpath='{range.items[*]}{.status.podIP}:6379 '
10.12.0.22:6379 10.12.1.17:6379 10.12.1.18:6379 10.12.0.23:6379 10.12.1.19:6379 10.12.0.24:6379
Hope that helps.
Update (06-Dec-2019):
have created a service LoadBalancer" and I am pretty sure I have given a correct "serviceName" under "spec". That LoadBalancer gave me an external Ip. After I created a redis sharded cluster, when I try to connect to that external IP using python, It was able to connect to only one node of redis and asks for other 5 nodes and eventually failed with "Timeout" error
I did the same and tested it Redis cluster on 3 nodes:
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP
redis-cluster-set-1-0 1/1 Running 0 20h 10.12.1.30
redis-cluster-set-1-1 1/1 Running 0 20h 10.12.0.29
redis-cluster-set-1-2 1/1 Running 0 20h 10.12.1.31
Created some data and tried to get it out from Redis cluster.
$ kubectl exec -it redis-cluster-set-1-0 -- redis-cli SET Australia Sydney
OK
$ kubectl exec -it redis-cluster-set-1-0 -- redis-cli GET Australia
"Sydney"
it worked so I tried to query the same data from another node in cluster
$ kubectl exec -it redis-cluster-set-1-2 -- redis-cli GET Australia
(error) MOVED 1738 10.12.1.30:6379
Redis cluster replied with Internal IP in MOVED
response.
I think that internal IP adress is the reason for that "timeOut" error.
At the same time it is possible to get data when logged into that pod directly
$ kubectl exec -it redis-cluster-set-1-2 -- bash
root@redis-cluster-set-1-2:/data# redis-cli GET Australia
(error) MOVED 1738 10.12.1.30:6379
root@redis-cluster-set-1-2:/data# redis-cli -h 10.12.1.30 GET Australia
"Sydney"
From this Github Issue it seems that redis doesn't proxy the connection through the Redis cluster, to correct node but rather simply passes the IP address of the appropriate server back to the client, which then initiates the connection directly.
(slight off-topic, but still worth mentioning). While checking on that I have found an [another document] that explains (https://cloud.google.com/memorystore/docs/redis/connect-redis-instance-gke) how to connect to a Redis instance from a Google Kubernetes Engine cluster (with LoadBalancer).