kubernetes StorageClass does not retain existing data

4/10/2019

My Kubernetes StorageClass volume doesn't retain existing data when the pod is deleted and deployed back with my postgresql database. When I delete the pod, the new pod is created but the database is empty.

I have followed variations of the different versions of the tutorials (https://kubernetes.io/docs/concepts/storage/persistent-volumes/) but nothing seems to work.

I paste all the YAML files cause the problem might be in the combination.

storage-google.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: spingular-pvc
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 7Gi
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
  zone: us-east4-a

jhipsterpress-postgresql.yml

apiVersion: v1
kind: Secret
metadata:
  name: jhipsterpress-postgresql
  namespace: default
  labels:
    app: jhipsterpress-postgresql
type: Opaque
data:
  postgres-password: NjY0NXJxd24=
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jhipsterpress-postgresql
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jhipsterpress-postgresql
    spec:
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: spingular-pvc
      containers:
      - name: postgres
        image: postgres:10.4
        env:
        - name: POSTGRES_USER
          value: jhipsterpress
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: jhipsterpress-postgresql
              key: postgres-password
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/
---
apiVersion: v1
kind: Service
metadata:
  name: jhipsterpress-postgresql
  namespace: default
spec:
  selector:
    app: jhipsterpress-postgresql
  ports:
  - name: postgresqlport
    port: 5432
  type: LoadBalancer

jhipsterpress-deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jhipsterpress
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jhipsterpress
      version: "v1"
  template:
    metadata:
      labels:
        app: jhipsterpress
        version: "v1"
    spec:
      initContainers:
        - name: init-ds
          image: busybox:latest
          command:
            - '/bin/sh'
            - '-c'
            - |
                while true
                do
                  rt=$(nc -z -w 1 jhipsterpress-postgresql 5432)
                  if [ $? -eq 0 ]; then
                    echo "DB is UP"
                    break
                  fi
                  echo "DB is not yet reachable;sleep for 10s before retry"
                  sleep 10
                done
      containers:
      - name: jhipsterpress-app
        image: galore/jhipsterpress
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: prod
        - name: SPRING_DATASOURCE_URL
          value: jdbc:postgresql://jhipsterpress-postgresql.default.svc.cluster.local:5432/jhipsterpress
        - name: SPRING_DATASOURCE_USERNAME
          value: jhipsterpress
        - name: SPRING_DATASOURCE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: jhipsterpress-postgresql
              key: postgres-password
        - name: JAVA_OPTS
          value: " -Xmx256m -Xms256m"
        resources:
          requests:
            memory: "256Mi"
            cpu: "500m"
          limits:
            memory: "512Mi"
            cpu: "1"
        ports:
        - name: http
          containerPort: 8080
        readinessProbe:
          httpGet:
            path: /management/health
            port: http
          initialDelaySeconds: 20
          periodSeconds: 15
          failureThreshold: 6
        livenessProbe:
          httpGet:
            path: /management/health
            port: http
          initialDelaySeconds: 120

jhipsterpress-service.yml

apiVersion: v1
kind: Service
metadata:
  name: jhipsterpress
  namespace: default
  labels:
    app: jhipsterpress
spec:
  selector:
    app: jhipsterpress
  type: LoadBalancer
  ports:
  - name: http
    port: 8080

When I included a Retain Policy I was getting this error:

@cloudshell:~ (academic-veld-230622)$ kubectl apply -f storage-google.yaml
error: error validating "storage-google.yaml": error validating data: 
ValidationError(PersistentVolumeClaim.spec): unknown field "persistentVolumeReclaimPolicy" in io.k8s.api.core.v1.PersistentVolumeClaimSpec; if you choose to ignore these errors, turn validation off with --validate=false

Please, if you know of a complete example on a public image that works (in postgresql, I can make it work with Mongo), I will really appreciate it.

Thanks to all.

-- Mike
kubernetes

1 Answer

4/10/2019

Note that for this to work you need to have your PVC dynamically provision a PV to satisfy its requirements, then there will be a permanent binding between the PVC and PV and every time your workload uses the PVC then it will use the same PV. Specifically indicated by this excerpt:

If a PV was dynamically provisioned for a new PVC, the loop will always bind that PV to the PVC

If in your case the Google Persistent Disk is being provisioned by the PVC, and you can verify that on GCP it's the same PV used every time, then it's probably an issue with the pod startup process where it's removing all the data. (Is there any reason why you are using /var/lib/postgresql/ vs /var/lib/postgresql?)

Also, persistentVolumeReclaimPolicy: Retain applies to a PV, not a PVC. For dynamically provisioned PVs the value is Delete. In your case, it wouldn't apply because your dynamically provisioned volume should be bound to your PVC. In other words, you are not reclaiming the volume.

Having said all that the recommended way to deploy a DB is using StatefulSets similar to this mysql example using a volumeClaimTemplate.

-- Rico
Source: StackOverflow