K
Q

How to expose a headless service for a StatefulSet cassandra cluster externally in Kubernetes

3/26/2019

I'm setting up multi node cassandra cluster in kubernetes (Azure AKS),Since this a headless service with statefull set pods without having a external IP. How can i connect my spark application with cassandra which is in kubernetes cluster

We have tried with cluster ip,ingress ip also but only single pod is getting up rest are failing.

I have 3 manifest:

  1. Service
  2. PersistentVolumeClaim
  3. StatefulSet
apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
spec:
  clusterIP: None
  ports:
    - port: 9042
  selector:
    app: cassandra


kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myvolume-disk-claim
spec:
  storageClassName: default
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi


apiVersion: "apps/v1"
kind: StatefulSet
metadata:
  name: cassandra
  labels:
    app: cassandra
spec:
  serviceName: cassandra
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
        - name: cassandra
          image: gcr.io/google-samples/cassandra:v13
          imagePullPolicy: Always
          ports:
            - containerPort: 7000
              name: intra-node
            - containerPort: 7001
              name: tls-intra-node
            - containerPort: 7199
              name: jmx
            - containerPort: 9042
              name: cql
          env:
            - name: CASSANDRA_SEEDS
              value: cassandra-0.cassandra.default.svc.cluster.local
            - name: MAX_HEAP_SIZE
              value: 256M
            - name: HEAP_NEWSIZE
              value: 100M
            - name: CASSANDRA_CLUSTER_NAME
              value: "Cassandra"
            - name: CASSANDRA_DC
              value: "DC1"
            - name: CASSANDRA_RACK
              value: "Rack1"
            - name: CASSANDRA_ENDPOINT_SNITCH
              value: GossipingPropertyFileSnitch
            - name: POD_IP
              valueFrom:
                fieldRef:
                   fieldPath: status.podIP
          readinessProbe:
            exec:
              command:
              - /bin/bash
              - -c
              - /ready-probe.sh
            initialDelaySeconds: 15
            timeoutSeconds: 5        
          volumeMounts:
          - mountPath: /var/lib/cassandra/data
            name: myvolume-disk-claim
  volumeClaimTemplates:
  - metadata:
      name: myvolume-disk-claim
    spec:
      storageClassName: default
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi

Expected Result:(public ip as external IP)

dspg@Digiteds28:$ kubectl get svc
NAME                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
kubernetes                ClusterIP   10.0.0.1     <none>        443/TCP    1h
cassandra                 ClusterIP   None        154.167.90.98  9042/TCP   1h



dspg@Digiteds28:$ kubectl get pod
NAME                READY   STATUS    RESTARTS   AGE
cassandra-0         1/1     Running   0          59m
cassandra-1         1/1     Running   0          58m
cassandra-2         1/1     Running   0          56m

Actual Output:

dspg@Digiteds28:$ kubectl get svc
NAME                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
kubernetes                ClusterIP   10.0.0.1     <none>        443/TCP    1h
cassandra                 ClusterIP   None         <none>        9042/TCP   1h



dspg@Digiteds28:$ kubectl get pod
NAME                READY   STATUS    RESTARTS   AGE
cassandra-0         1/1     Running   0          59m
cassandra-1         1/1     Running   0          58m
cassandra-2         1/1     Running   0          56m

Now this doesnot include external IP to connect to application.

-- suraj1287
apache-spark
cassandra
kubernetes

Similar Questions

Rancher static IP for containers
Connection issue between services in kubernetes
ZuulProperties return no routes
Why Kubernetes HPA convert custom metric?
Failed to pull image repository does not exist or may require &#39;docker login&#39;:

2 Answers

3/27/2019
apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra-ext
spec:
  type: LoadBalancer
  ports:
    - port: 9042
  selector:
    app: cassandra

Using the additonal service gave me a External IP and because if which i started connecting my spark or devcenter application to cassandra cluster.

-- suraj1287
Source: StackOverflow
3/26/2019

It depends on what exactly you are trying to do. If you need an external IP then in general you'd need to create an additional Service object (probably type: LoadBalancer) like this:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra-ext
spec:
  type: LoadBalancer
  ports:
    - port: 9042
  selector:
    app: cassandra

If you need to reach it from within the cluster then use the DNS name cassandra-0.cassandra.default from the other pod (if the StatefulSet was deployed in the default namespace)

-- Janos Lenart
Source: StackOverflow