minikube mongodb statefulset with persistent volume config for 3 replicas

9/29/2019

Pod fails to start and I am having a tough time debugging it with my limited experience. I have also mounted a volume from my local machine to the minikube vm like so: minikube start --cpus 4 --memory 8192 --mount-string="/data/minikube:/data" --mount

Any help would be much appreciated.

Storage Class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: db-sc
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

Persistent Volume:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: db-pv
spec:
  storageClassName: db-sc
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  accessModes:
    - ReadWriteOnce
  local:
    path: "/data"
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - minikube

StatefulSet:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: test-db
  labels:
    app: test-db
spec:
  serviceName: test-db
  replicas: 3
  selector:
    matchLabels:
      app: test-db
      replicaset: MainRepSet
  template:
    metadata:
      labels:
        app: test-db
        replicaset: MainRepSet
    spec:
      containers:
        - name: test-db
          image: mongo:3.4
          command:
            - "numactl"
            - "--interleave=all"
            - "mongod"
            - "--bind_ip"
            - "0.0.0.0"
            - "--replSet"
            - "MainRepSet"
            - "--auth"
            - "--clusterAuthMode"
            - "keyFile"
            - "--keyFile"
            - "/etc/secrets-volume/internal-auth-db-keyfile"
            - "--setParameter"
            - "authenticationMechanisms=SCRAM-SHA-1"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: secrets-volume
              readOnly: true
              mountPath: /etc/secrets-volume
            - name: db-pvc
              mountPath: /data/db
          resources:
            requests:
              cpu: 150m
              memory: 150Mi
            limits:
              cpu: 400m
              memory: 400Mi
      volumes:
        - name: secrets-volume
          secret:
            secretName: db-user
            defaultMode: 256
  volumeClaimTemplates:
    - metadata:
        name: db-pvc
      spec:
        storageClassName: db-sc
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 500Mi

Service:

apiVersion: v1
kind: Service
metadata:
  name: test-db
  labels:
    name: test-db
spec:
  selector:
    app: test-db
  clusterIP: None
  ports:
    - port: 27017

Pod logs from test-db-0:

Loading...2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=test-db-0
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] db version v3.4.23
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] git version: 324017ede1dbb1c9554dd2dceb15f8da3c59d0e8
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] modules: none
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] build environment:
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten]     distmod: ubuntu1604
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten]     distarch: x86_64
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2019-09-29T20:10:16.413+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "0.0.0.0" }, replication: { replSet: "MainRepSet" }, security: { authorization: "enabled", clusterAuthMode: "keyFile", keyFile: "/etc/secrets-volume/internal-auth-db-keyfile" }, setParameter: { authenticationMechanisms: "SCRAM-SHA-1" } }
2019-09-29T20:10:16.416+0000 I STORAGE  [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock Unknown error 526 Is a mongod instance already running?, terminating
2019-09-29T20:10:16.416+0000 I NETWORK  [initandlisten] shutdown: going to close listening sockets...
2019-09-29T20:10:16.416+0000 I NETWORK  [initandlisten] shutdown: going to flush diaglog...
2019-09-29T20:10:16.417+0000 I CONTROL  [initandlisten] now exiting
2019-09-29T20:10:16.417+0000 I CONTROL  [initandlisten] shutting down with code:100
-- kfcobrien
kubernetes-statefulset
minikube
mongodb-replica-set
persistent-volume-claims
persistent-volumes

0 Answers