Is there any less hacky way of passing an Ordinal Index to StatefulSet environment variable?

5/24/2019

I'm trying to run a zookeeper ensemble and am having an issue passing a unique ID as envrionment varible ZOO_MY_ID as required by official zookeeeper image found here.

I've tried reading about this and found similar overflow questions but none seems to be working.

kubernetes statefulsets index/ordinal exposed in template Is there a way to get ordinal index of a pod with in kubernetes statefulset configuration file?

For some reason, I am still seeing the ID for all servers to be the default id of 1

2019-05-24 01:38:31,648 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@847] - Notification time out: 60000
2019-05-24 01:38:31,649 [myid:1] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@347] - Have smaller server identifier, so dropping the connection: (2, 1)
2019-05-24 01:38:31,649 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@595] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2019-05-24 01:38:31,649 [myid:1] - INFO  [/0.0.0.0:3888:QuorumCnxManager$Listener@743] - Received connection request /10.24.1.64:37382
2019-05-24 01:38:31,650 [myid:1] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@1025] - Connection broken for id 1, my id = 1, error = 
java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:392)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:1010)
2019-05-24 01:38:31,651 [myid:1] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@1028] - Interrupting SendWorker

Running the following command shows that no ID is passed however I am using the hacky way shown here: https://stackoverflow.com/a/48086813/5813215

kubectl exec -it zoo-2 -n kafka-dev printenv | grep "ZOO_"

ZOO_USER=zookeeper
ZOO_CONF_DIR=/conf
ZOO_DATA_DIR=/data
ZOO_DATA_LOG_DIR=/datalog
ZOO_LOG_DIR=/logs
ZOO_PORT=2181
ZOO_TICK_TIME=2000
ZOO_INIT_LIMIT=5
ZOO_SYNC_LIMIT=2
ZOO_AUTOPURGE_PURGEINTERVAL=0
ZOO_AUTOPURGE_SNAPRETAINCOUNT=3
ZOO_MAX_CLIENT_CNXNS=60
-- Ani Aggarwal
google-kubernetes-engine
kubernetes
kubernetes-statefulset

2 Answers

7/1/2019

I am not sure if it was resolved so:

As mentioned in the StatefulSets concept, the Pods in a StatefulSet have a sticky, unique identity. This identity is based on a unique ordinal index that is assigned to each Pod by the StatefulSet controller.

You can find an example here.

For example You can modify your statefulSet spec. by adding:

   env:
      - name: MY_POD_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.name

You can parse the index out of that.

More information and discussion about this particular topic you can find here

Hope this help.

-- Hanx
Source: StackOverflow

5/24/2019
apiVersion: v1
kind: Service
metadata:
  name: zk-svc
  labels:
    app: zk-svc
spec:
  ports:
  - port: 2888
    name: server
  - port: 3888
    name: leader-election
  clusterIP: None
  selector:
    app: zk
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: zk-cm
data:
  jvm.heap: "1G"
  tick: "2000"
  init: "10"
  sync: "5"
  client.cnxns: "60"
  snap.retain: "3"
  purge.interval: "0"
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: zk-pdb
spec:
  selector:
    matchLabels:
      app: zk
  minAvailable: 2
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: zk
spec:
  serviceName: zk-svc
  replicas: 3
  template:
    metadata:
      labels:
        app: zk
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values: 
                    - zk
              topologyKey: "kubernetes.io/hostname"
      containers:
      - name: k8szk
        imagePullPolicy: Always
        image: gcr.io/google_samples/k8szk:v3
        resources:
          requests:
            memory: "2Gi"
            cpu: "500m"
        ports:
        - containerPort: 2181
          name: client
        - containerPort: 2888
          name: server
        - containerPort: 3888
          name: leader-election
        env:
        - name : ZK_REPLICAS
          value: "3"
        - name : ZK_HEAP_SIZE
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: jvm.heap
        - name : ZK_TICK_TIME
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: tick
        - name : ZK_INIT_LIMIT
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: init
        - name : ZK_SYNC_LIMIT
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: tick
        - name : ZK_MAX_CLIENT_CNXNS
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: client.cnxns
        - name: ZK_SNAP_RETAIN_COUNT
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: snap.retain
        - name: ZK_PURGE_INTERVAL
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: purge.interval
        - name: ZK_CLIENT_PORT
          value: "2181"
        - name: ZK_SERVER_PORT
          value: "2888"
        - name: ZK_ELECTION_PORT
          value: "3888"
        command:
        - sh
        - -c
        - zkGenConfig.sh && zkServer.sh start-foreground
        readinessProbe:
          exec:
            command:
            - "zkOk.sh"
          initialDelaySeconds: 10
          timeoutSeconds: 5
        livenessProbe:
          exec:
            command:
            - "zkOk.sh"
          initialDelaySeconds: 10
          timeoutSeconds: 5
        volumeMounts:
        - name: datadir
          mountPath: /var/lib/zookeeper
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 10Gi
-- Harsh Manvar
Source: StackOverflow