Postgres pod crashed after running init script, not able to restart, the last line of the log says ": not foundbin/docker-entrypoint.sh: line 176: exec:"
I have created a configmap named idp-db-init-script with the make-db.sh file, and mounted this inside the pod at /docker-entrypoint-initdb.d
make-db.sh
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "postgres" --dbname "postgres" <<-EOSQL
CREATE USER keycloak;
CREATE DATABASE keycloak;
GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak;
EOSQL
Created configmap with command line
kubectl create configmap idp-db-init-script --from-file=/home/ubuntu/make_db.sh
YAML file
apiVersion: apps/v1
kind: Deployment
metadata:
name: idp-db
spec:
selector:
matchLabels:
app: idp-db
replicas: 1
template:
metadata:
labels:
app: idp-db
spec:
containers:
- image: "postgres"
name: idp-db
ports:
- containerPort: 5432
name: idp-db
protocol: TCP
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: db-data
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: init-script
volumes:
- name: init-script
configMap:
name: idp-db-init-script
defaultMode: 0744
- name: db-data
emptyDir: {}
Result log via kubectl logs -f
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
****************************************************
WARNING: No password has been set for the database.
This will allow anyone with access to the
Postgres port to access your database. In
Docker's default configuration, this is
effectively any other container on the same
system.
Use "-e POSTGRES_PASSWORD=password" to set
it in "docker run".
****************************************************
waiting for server to start....2019-04-06 11:16:39.920 UTC [43] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-04-06 11:16:39.962 UTC [44] LOG: database system was shut down at 2019-04-06 11:16:39 UTC
2019-04-06 11:16:39.975 UTC [43] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/make_db.sh
CREATE ROLE
CREATE DATABASE
GRANT
waiting for server to shut down...2019-04-06 11:16:40.287 UTC [43] LOG: received fast shutdown request
.2019-04-06 11:16:40.290 UTC [43] LOG: aborting any active transactions
2019-04-06 11:16:40.291 UTC [43] LOG: background worker "logical replication launcher" (PID 50) exited with exit code 1
2019-04-06 11:16:40.293 UTC [45] LOG: shutting down
2019-04-06 11:16:40.358 UTC [43] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
: not foundbin/docker-entrypoint.sh: line 176: exec:
Describe pod result:
Name: idp-db-7d7d56b7cd-tz5bn
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: ip-10-2-3-179.ca-central-1.compute.internal/10.2.3.179
Start Time: Sat, 06 Apr 2019 11:05:19 +0000
Labels: app=idp-db
pod-template-hash=7d7d56b7cd
Annotations: <none>
Status: Running
IP: 192.168.23.199
Controlled By: ReplicaSet/idp-db-7d7d56b7cd
Containers:
idp-db:
Container ID: docker://325889248dc0b80d5f2f0e8c945bb314563a35a767ef0b379b683c4d2b1a1dd9
Image: postgres
Image ID: docker-pullable://postgres@sha256:e002abfaf7b0f1015b7510f66b709e2bacbecc7762c428634c27a3c5e58ffbae
Port: 5432/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 127
Started: Sat, 06 Apr 2019 11:16:38 +0000
Finished: Sat, 06 Apr 2019 11:16:40 +0000
Ready: False
Restart Count: 7
Environment: <none>
Mounts:
/docker-entrypoint-initdb.d from init-script (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-ckwk8 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
init-script:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: idp-db-init-script
Optional: false
db-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
default-token-ckwk8:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-ckwk8
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 13m default-scheduler Successfully assigned default/idp-db-7d7d56b7cd-tz5bn to ip-10-2-3-179.ca-central-1.compute.internal
Normal Created 12m (x4 over 13m) kubelet, ip-10-2-3-179.ca-central-1.compute.internal Created container
Normal Started 12m (x4 over 13m) kubelet, ip-10-2-3-179.ca-central-1.compute.internal Started container
Normal Pulling 11m (x5 over 13m) kubelet, ip-10-2-3-179.ca-central-1.compute.internal pulling image "postgres"
Normal Pulled 11m (x5 over 13m) kubelet, ip-10-2-3-179.ca-central-1.compute.internal Successfully pulled image "postgres"
Warning BackOff 3m21s (x47 over 13m) kubelet, ip-10-2-3-179.ca-central-1.compute.internal Back-off restarting failed container
As per @Sudhakar comment the issue has be resolved by recreating initial script, which was probably affected by some special character or another incorrect formatting case.