Expose a redis cluster - with a kubernetes statefulset to the internet

7/23/2019

I created a statefulset that deploys a redis image to GCP on kubernetes. The challenge I am having is exposing it using a single domain name. Such that the pods can be accessed in the following order - redis.com/first, redis.com/second, redis.com/third

Simple diagram to show the kind of structure I want to implement

here are the YAML files

Statefulset

kind: StatefulSet
metadata:
  name: app-redis
spec:
  selector:
    matchLabels:
      app: apprenticeship-redis
  serviceName: 'redis-service'
  replicas: 3
  template:
    metadata:
      labels:
        app: app-redis
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: app-redis
          image: redis
          args:
            - /etc/redis/redis.conf
          volumeMounts:
            - mountPath: /etc/redis
              name: redis-config
              readOnly: false
            - name: redis-storage
              mountPath: /data
              readOnly: false
          resources:
            requests:
              cpu: 50m
              memory: 128Mi
            limits:
              cpu: 150m
              memory: 256Mi
          ports:
            - containerPort: 6379
              name: redis
          livenessProbe:
            exec:
              command: ['redis-cli', 'ping']
            initialDelaySeconds: 30
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 2
      volumes:
        - name: redis-config
          configMap:
            name: redis-config
  volumeClaimTemplates:
    - metadata:
        name: redis-storage
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi

Headless service

apiVersion: v1
kind: Service
metadata:
  labels:
    app: app-redis
  name: redis-service
  namespace: default
spec:
  ports:
    - name: server-port
      port: 80
      protocol: TCP
      targetPort: 6379
  clusterIP: None
  selector:
    statefulset.kubernetes.io/pod-name: app-redis-0

Loadbalancer

apiVersion: v1
kind: Service
metadata:
  labels:
    app: redis-service
  name: app-redis
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 6379
  selector:
    app: app-redis
  type: LoadBalancer
  loadBalancerIP: xx.xx.xx.xxx
status:
  loadBalancer:
    ingress:
    - ip: xx.xx.xx.xxx

Config map

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
  namespace: default
data:
  redis.conf: |
    dbfilename "dump.rdb"
    dir /data
    save 3600 1
    save 300 10
    save 60 100
    appendonly yes
    appendfilename "appendonly.aof"

Storage class

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: redis-storage
provisioner: kubernetes.io/gce-pd

Ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: redis-ingress
  annotations:
    kubernetes.io/ingress.class: 'nginx'
    nginx.ingress.kubernetes.io/force-ssl-redirect: 'false'
spec:
  rules:
    - host: app-redis.tk
      http:
        paths:
          - path: /
            backend:
              serviceName: app-redis
              servicePort: 80
-- David Essien
devops
kubernetes
kubernetes-statefulset
redis

1 Answer

8/5/2019

Each pod in the StatefulSet will need to have a service linking to it.

This service will need to be created with:

  selector:
    statefulset.kubernetes.io/pod-name: <POD_NAME>

Then you will be able to set ingress and use it to redirect traffic based on path:

...
spec:
  rules:
  - http:
      paths:
        - path: /app-redis-0
          backend:
            serviceName: redis-service-0
            servicePort: 6379
        - path: /app-redis-1
          backend:
            serviceName: redis-service-1
            servicePort: 6379
        - path: /app-redis-2
          backend:
            serviceName: redis-service-2
            servicePort: 6379            
...            

You can read about Exposing StatefulSets in Kubernetes and Kubernetes NodePort vs LoadBalancer vs Ingress? When should I use what?

-- Crou
Source: StackOverflow