Production Redis cluster with sharding in Kubernetes

10/28/2019

I have tried Redis stable 'helm' chart to deploy a Redis cluster with 1 master and 3 slaves that replicates data written to master. But it is a single point of failure - I deleted the master, no new pod of master was recreated. Also, the chart does not support data partitions (sharding).

EDIT: I have created a Redis cluster using helm redis-ha chart, but there is no option to have sharding.

Aren't there Redis helm charts to deploy a production ready HA cluster that supports partitions (sharding)? Can you point me to resources I can use to setup a manageable Redis cluster? Primarily, my Redis is used for data caching, message processing & streaming.

-- Vijay Veeraraghavan
kubernetes
redis

2 Answers

10/29/2019

Here's a fine tutorial about setting up a 3 master / 3 slaves redis cluster with partitioning. It's targetted at Rancher, but that's optional, I just tested it on Azure Kubernetes Service and it works fine.

First, apply this yaml (ConfigMap, StatefulSet with 6 replicas, Service):

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-cluster
data:
  update-node.sh: |
    #!/bin/sh
    REDIS_NODES="/data/nodes.conf"
    sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${POD_IP}/" ${REDIS_NODES}
    exec "$@"
  redis.conf: |+
    cluster-enabled yes
    cluster-require-full-coverage no
    cluster-node-timeout 15000
    cluster-config-file /data/nodes.conf
    cluster-migration-barrier 1
    appendonly yes
    protected-mode no
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
spec:
  serviceName: redis-cluster
  replicas: 6
  selector:
    matchLabels:
      app: redis-cluster
  template:
    metadata:
      labels:
        app: redis-cluster
    spec:
      containers:
      - name: redis
        image: redis:5.0.1-alpine
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
      volumes:
      - name: conf
        configMap:
          name: redis-cluster
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: redis-cluster
spec:
  type: ClusterIP
  ports:
  - port: 6379
    targetPort: 6379
    name: client
  - port: 16379
    targetPort: 16379
    name: gossip
  selector:
    app: redis-cluster

And the next step is to run this script to form the cluster (you will have to enter "yes" once interactively):

kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
for x in $(seq 0 5); do echo "redis-cluster-$x"; kubectl exec redis-cluster-$x -- redis-cli role; echo; done
-- Markus Dresch
Source: StackOverflow

10/28/2019

You need Redis Sentinel.

If helm is an option, these links can be of help:

https://github.com/helm/charts/tree/master/stable/redis (notice the sentinel-related configuration parameters)

https://github.com/helm/charts/tree/master/stable/redis-ha

-- apisim
Source: StackOverflow