Expose every pod in Redis cluster in Kubernetes

7/6/2018

I'm trying to setup Redis cluster in Kubernetes. The major requirement is that all of nodes from Redis cluster have to be available from outside of Kubernetes. So clients can connect every node directly. But I got no idea how to configure service that way.

Basic config of cluster right now. It's ok for services into k8s but no full access from outside.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: redis-cluster
      labels:
        app: redis-cluster
    data:
      redis.conf: |+
        cluster-enabled yes
        cluster-require-full-coverage no
        cluster-node-timeout 15000
        cluster-config-file /data/nodes.conf
        cluster-migration-barrier 1
        appendonly no
        protected-mode no
    ---
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.alpha.kubernetes.io/tolerate-unready-endpoints: "false"
      name: redis-cluster
      labels:
        app: redis-cluster
    spec:
      type: NodePort
      ports:
      - port: 6379
        targetPort: 6379
        name: client
      - port: 16379
        targetPort: 16379
        name: gossip
      selector:
        app: redis-cluster
    ---
    apiVersion: apps/v1beta1
    kind: StatefulSet
    metadata:
      name: redis-cluster
      labels:
        app: redis-cluster
    spec:
      serviceName: redis-cluster
      replicas: 6
      template:
        metadata:
          labels:
            app: redis-cluster
        spec:
          hostNetwork: true
          containers:
          - name: redis-cluster
            image: redis:4.0.10
            ports:
            - containerPort: 6379
              name: client
            - containerPort: 16379
              name: gossip
            command: ["redis-server"]
            args: ["/conf/redis.conf"]
            readinessProbe:
              exec:
                command:
                - sh
                - -c
                - "redis-cli -h $(hostname) ping"
              initialDelaySeconds: 15
              timeoutSeconds: 5
            livenessProbe:
              exec:
                command:
                - sh
                - -c
                - "redis-cli -h $(hostname) ping"
              initialDelaySeconds: 20
              periodSeconds: 3
            volumeMounts:
            - name: conf
              mountPath: /conf
              readOnly: false
          volumes:
          - name: conf
            configMap:
              name: redis-cluster
              items: 
              - key: redis.conf
                path: redis.conf
-- Shtlzut
kubernetes
redis
redis-cluster

1 Answer

7/7/2018

Given:

    spec:
      hostNetwork: true
      containers:
      - name: redis-cluster
        ports:
        - containerPort: 6379
          name: client 

It appears that your StatefulSet is misconfigured, since if hostNetwork is true, you have to provide hostPort, and that value should match containerPort, according to the PodSpec docs:

hostPort integer - Number of port to expose on the host. If specified, this must be a valid port number, 0 < x < 65536. If HostNetwork is specified, this must match ContainerPort.

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#containerport-v1-core

-- mdaniel
Source: StackOverflow