(Kubernetes) - Database empty when restarting server

10/30/2017

I have a postgres database running on a pod. It runs without trouble, and the data is there. There is a persistent volume, whose purpose is to keep the data regardless of what happens with the pods.

However, when I restart the server, the database appears empty. However, the volumes are mounted and the files are in the filesystem, both in the server and in the pod.

No relations/tables or whatsoever appear on my DB:

(Inside the pod)

psql (9.6.2)
Type "help" for help.

information_system=# \d

ls on my machine, where the volume is:

blas@server:~$ sudo ls -l  /tmp/data/postgres/
total 120
drwx------ 6 999 docker  4096 Oct 30 11:21 base
drwx------ 2 999 docker  4096 Oct 30 11:22 global
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_clog
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_commit_ts
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_dynshmem
-rw------- 1 999 docker  4490 Oct 30 11:21 pg_hba.conf
-rw------- 1 999 docker  1636 Oct 30 11:21 pg_ident.conf
drwx------ 4 999 docker  4096 Oct 30 11:21 pg_logical
drwx------ 4 999 docker  4096 Oct 30 11:21 pg_multixact
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_notify
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_replslot
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_serial
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_snapshots
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_stat
drwx------ 2 999 docker  4096 Oct 30 11:53 pg_stat_tmp
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_subtrans
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_tblspc
drwx------ 2 999 docker  4096 Oct 30 11:21 pg_twophase
-rw------- 1 999 docker     4 Oct 30 11:21 PG_VERSION
drwx------ 3 999 docker  4096 Oct 30 11:21 pg_xlog
-rw------- 1 999 docker    88 Oct 30 11:21 postgresql.auto.conf
-rw------- 1 999 docker 22205 Oct 30 11:21 postgresql.conf
-rw------- 1 999 docker    37 Oct 30 11:21 postmaster.opts
-rw------- 1 999 docker    85 Oct 30 11:21 postmaster.pid

ls on the pod, where the volume is mounted:

root@information-system-deployment-5dccfcb7c9-54trz:/var/lib/postgresql/data# ls -l
total 120
drwx------ 6 postgres postgres  4096 Oct 30 11:21 base
drwx------ 2 postgres postgres  4096 Oct 30 11:22 global
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_clog
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_commit_ts
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_dynshmem
-rw------- 1 postgres postgres  4490 Oct 30 11:21 pg_hba.conf
-rw------- 1 postgres postgres  1636 Oct 30 11:21 pg_ident.conf
drwx------ 4 postgres postgres  4096 Oct 30 11:21 pg_logical
drwx------ 4 postgres postgres  4096 Oct 30 11:21 pg_multixact
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_notify
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_replslot
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_serial
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_snapshots
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_stat
drwx------ 2 postgres postgres  4096 Oct 30 11:55 pg_stat_tmp
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_subtrans
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_tblspc
drwx------ 2 postgres postgres  4096 Oct 30 11:21 pg_twophase
-rw------- 1 postgres postgres     4 Oct 30 11:21 PG_VERSION
drwx------ 3 postgres postgres  4096 Oct 30 11:21 pg_xlog
-rw------- 1 postgres postgres    88 Oct 30 11:21 postgresql.auto.conf
-rw------- 1 postgres postgres 22205 Oct 30 11:21 postgresql.conf
-rw------- 1 postgres postgres    37 Oct 30 11:21 postmaster.opts
-rw------- 1 postgres postgres    85 Oct 30 11:21 postmaster.pid

Deployment:

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: information-system-deployment
spec:
  selector:
    matchLabels:
      app: information-system-deployment
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: information-system-deployment
    spec:

      containers:
      - image: my-registry:5000/information-system-db
        name: information-system-db
        env:
        - name: POSTGRES_DB
          value: information_system
        - name: POSTGRES_USER
          valueFrom:
            secretKeyRef:
              name: information-system-db-secret
              key: username
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: information-system-db-secret
              key: password
        ports:
        - containerPort: 5432
          name: is-db
        volumeMounts:
        - name: information-system-db-pv-volume
          mountPath: /var/lib/postgresql/data

      - image: my-registry:5000/information-system-test:latest
        name: information-system
        ports:
        - containerPort: 5010
          name: is
        command: ["bash", "-c", "python main.py"]

    volumes:
    - name: information-system-db-pv-volume
      persistentVolumeClaim:
        claimName: information-system-db-claim

Persistent volume:

 kind: PersistentVolume
  apiVersion: v1
  metadata:
    name: information-system-db-pv-volume
    labels:
      type: local
  spec:
    storageClassName: manual
    capacity:
      storage: 10Gi
    accessModes:
      - ReadWriteOnce
    hostPath:
      path: "/tmp/data/postgres"

Persistent storage claim

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: information-system-db-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
-- blasrodri
kubernetes
postgresql

1 Answer

10/30/2017

I don't think you should be using /tmp/* directories for your hostpath.

/tmp/* directories are cleaned on every reboot: https://askubuntu.com/questions/20783/how-is-the-tmp-directory-cleaned-up

-- iamnat
Source: StackOverflow