I'm using the Kubernetes Postgres StateulSet example repo here by Arianit Uka: https://github.com/arianitu/postgres-statefulset
with minikube and my pod container isn't coming up. The secrets have been applied and it seems to have the POSTGRES_PASSWORD
env var just fine. There's no password=
in the repo's code so I'm lost where the issue may be. Checking the logs gives this:
> $ k logs postgres-0 --all-containers=true ⬡ 8.11.4 [±hauser ●●]
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data/pgdata ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data/pgdata -l logfile start
waiting for server to start....2019-08-25 07:22:31.295 UTC [41] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-08-25 07:22:31.307 UTC [42] LOG: database system was shut down at 2019-08-25 07:22:31 UTC
2019-08-25 07:22:31.310 UTC [41] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/create-dev-db.sh
CREATE DATABASE
GRANT
/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/create-replica-user.sh
CREATE ROLE
2019-08-25 07:22:31.751 UTC [41] LOG: received fast shutdown request
waiting for server to shut down....2019-08-25 07:22:31.752 UTC [41] LOG: aborting any active transactions
2019-08-25 07:22:31.753 UTC [41] LOG: worker process: logical replication launcher (PID 48) exited with exit code 1
2019-08-25 07:22:31.755 UTC [43] LOG: shutting down
2019-08-25 07:22:31.774 UTC [41] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2019-08-25 07:22:31.862 GMT [1] LOG: skipping missing configuration file "/etc/replica.conf"
2019-08-25 07:22:31.862 GMT [1] LOG: skipping missing configuration file "/etc/replica.conf"
2019-08-25 07:22:31.865 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2019-08-25 07:22:31.865 UTC [1] LOG: listening on IPv6 address "::", port 5432
2019-08-25 07:22:31.869 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-08-25 07:22:31.888 UTC [68] LOG: database system was shut down at 2019-08-25 07:22:31 UTC
2019-08-25 07:22:31.893 UTC [1] LOG: database system is ready to accept connections
2019-08-25 07:23:20.352 UTC [75] FATAL: role "password=" does not exist
2019-08-25 07:23:23.595 UTC [82] FATAL: role "password=" does not exist
Here's what the minikube dashboard looks like:
@Rahman, I took out the password secrets and am using just plain values for now in both statefulset-master.yml
and statefulset-replica.yml
:
env:
...
- name: POSTGRES_PASSWORD
value: master-password
- name: REPLICATION_PASSWORD
value: replica-password
I've also commented out the replica section in service.yml
as I'm only interesting in running a single pod/database at the moment:
service.yml
apiVersion: v1
kind: Service
metadata:
labels:
app: postgres
name: postgres
spec:
type: ClusterIP
ports:
- name: postgres
port: 5432
protocol: TCP
targetPort: 5432
selector:
app: postgres
# ---
# apiVersion: v1
# kind: Service
# metadata:
# labels:
# app: postgres-replica
# name: postgres-replica
# spec:
# type: ClusterIP
# ports:
# - name: postgres-replica
# port: 5432
# protocol: TCP
# targetPort: 5432
# selector:
# app: postgres-replica
# ---
statefulset-master.yml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
spec:
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: postgres
serviceName: postgres
replicas: 1
template:
metadata:
labels:
app: postgres
spec:
volumes:
- name: postgres-config
configMap:
name: postgres
# - name: shared
# emptyDir: {}
terminationGracePeriodSeconds: 10
containers:
- name: postgres
image: postgres:10.5
args: ['-c', 'config_file=/etc/postgres.conf', '-c', 'hba_file=/etc/pg_hba.conf']
imagePullPolicy: IfNotPresent
ports:
- name: postgres
containerPort: 5432
protocol: TCP
resources:
requests:
cpu: 100m
memory: 256Mi
env:
- name: POSTGRES_USER
value: postgres
- name: PGUSER
value: postgres
- name: POSTGRES_DB
value: postgres
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POSTGRES_PASSWORD
value: master-password
# valueFrom:
# secretKeyRef:
# key: password
# name: postgres
- name: REPLICATION_PASSWORD
value: replica-password
# valueFrom:
# secretKeyRef:
# key: replicaPassword
# name: postgres
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
livenessProbe:
exec:
command:
- sh
- -c
- exec pg_isready --host $POD_IP
failureThreshold: 6
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
exec:
command:
- sh
- -c
- exec pg_isready --host $POD_IP
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 3
volumeMounts:
- mountPath: /var/lib/postgresql/data/pgdata
name: postgres
subPath: postgres-db
- name: postgres-config
mountPath: /etc/postgres.conf
subPath: postgres.conf
- name: postgres-config
mountPath: /etc/master.conf
subPath: master.conf
- name: postgres-config
mountPath: /etc/pg_hba.conf
subPath: pg_hba.conf
- name: postgres-config
mountPath: /docker-entrypoint-initdb.d/create-replica-user.sh
subPath: create-replica-user.sh
- name: postgres-config
mountPath: /docker-entrypoint-initdb.d/create-dev-db.sh
subPath: create-dev-db.sh
# - name: shared
# mountPath: /User/Shared
- name: hauser
image: mikeumus/hauser
# volumeMounts:
# - name: shared
# mountPath: /User/Shared
volumeClaimTemplates:
- metadata:
name: postgres
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "standard"
resources:
requests:
storage: 3Gi
statefulset-replica.yml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-replica
spec:
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: postgres-replica
serviceName: postgres-replica
replicas: 1
template:
metadata:
labels:
app: postgres-replica
spec:
volumes:
- name: postgres-config
configMap:
name: postgres
terminationGracePeriodSeconds: 10
initContainers:
- name: setup-replica-data-directory
image: postgres:10.5
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
key: replicaPassword
name: postgres
command:
- sh
- -c
- |
if [ -z "$(ls -A /var/lib/postgresql/data/pgdata)" ]; then
echo "Running pg_basebackup to catch up replication server...";
pg_basebackup -R -h postgres -D /var/lib/postgresql/data/pgdata -P -U replication;
chown -R postgres:postgres $PGDATA;
else
echo "Skipping pg_basebackup because directory is not empty";
fi
volumeMounts:
- mountPath: /var/lib/postgresql/data/pgdata
name: postgres-replica
subPath: postgres-db
containers:
- name: postgres-replica
image: postgres:10.5
args: ['-c', 'config_file=/etc/postgres.conf']
imagePullPolicy: IfNotPresent
ports:
- name: postgres-rep
containerPort: 5432
protocol: TCP
resources:
requests:
cpu: 100m
memory: 256Mi
env:
- name: POSTGRES_USER
value: postgres
- name: PGUSER
value: postgres
- name: POSTGRES_DB
value: postgres
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POSTGRES_PASSWORD
value: master-password
# valueFrom:
# secretKeyRef:
# key: password
# name: postgres
- name: REPLICATION_PASSWORD
value: replica-password
# valueFrom:
# secretKeyRef:
# key: replicaPassword
# name: postgres
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
livenessProbe:
exec:
command:
- sh
- -c
- exec pg_isready --host $POD_IP
failureThreshold: 6
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
exec:
command:
- sh
- -c
- exec pg_isready --host $POD_IP
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 3
volumeMounts:
- mountPath: /var/lib/postgresql/data/pgdata
name: postgres-replica
subPath: postgres-db
- name: postgres-config
mountPath: /etc/postgres.conf
subPath: postgres.conf
- name: postgres-config
mountPath: /etc/replica.conf
subPath: replica.conf
volumeClaimTemplates:
- metadata:
name: postgres-replica
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "standard"
resources:
requests:
storage: 3Gi
config/create_configmap.sh
kubectl create configmap postgres --from-file=postgres.conf --from-file=master.conf --from-file=replica.conf --from-file=pg_hba.conf --from-file=create-replica-user.sh --from-file=create-dev-db.sh
config/create-replica-user.sh
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE ROLE replication WITH REPLICATION PASSWORD '$REPLICATION_PASSWORD' LOGIN
EOSQL
Here's the order I start/apply resources:
minikube start
cd config
I get some missing error unless I'm in config dir when I do this. ./create_configmap.sh
cd ..
k apply -f ./config/secret.yml
k apply -f ./service.yml
k apply -f ./statefulset-master.yml
Link to my fork of postgres-statefulset
repo: https://gitlab.com/mikeumus/postgres-statefulset
It wasn't the postgres-stateful
code that was the problem. The "password=
" code was in the Hauser code I added as another container to statefulset-master.yml