Postgres pod crashing with error "not foundbin/docker-entrypoint.sh: line 176: exec:"

4/6/2019

Postgres pod crashed after running init script, not able to restart, the last line of the log says ": not foundbin/docker-entrypoint.sh: line 176: exec:"

I have created a configmap named idp-db-init-script with the make-db.sh file, and mounted this inside the pod at /docker-entrypoint-initdb.d

make-db.sh

#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "postgres" --dbname "postgres" <<-EOSQL
     CREATE USER keycloak;
     CREATE DATABASE keycloak;
     GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak;
EOSQL

Created configmap with command line

kubectl create configmap idp-db-init-script --from-file=/home/ubuntu/make_db.sh

YAML file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: idp-db
spec:
  selector:
    matchLabels:
      app: idp-db
  replicas: 1
  template:
    metadata:
      labels:
        app: idp-db
    spec:
      containers:
        - image: "postgres"
          name: idp-db
          ports:
            - containerPort: 5432
              name: idp-db
              protocol: TCP
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: db-data
          volumeMounts:
            - mountPath: /docker-entrypoint-initdb.d
              name: init-script
      volumes:
        - name: init-script
          configMap:
            name: idp-db-init-script
          defaultMode: 0744
        - name: db-data
          emptyDir: {}

Result log via kubectl logs -f

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start


WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
****************************************************
WARNING: No password has been set for the database.
         This will allow anyone with access to the
         Postgres port to access your database. In
         Docker's default configuration, this is
         effectively any other container on the same
         system.

         Use "-e POSTGRES_PASSWORD=password" to set
         it in "docker run".
****************************************************
waiting for server to start....2019-04-06 11:16:39.920 UTC [43] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-04-06 11:16:39.962 UTC [44] LOG:  database system was shut down at 2019-04-06 11:16:39 UTC
2019-04-06 11:16:39.975 UTC [43] LOG:  database system is ready to accept connections
 done
server started

/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/make_db.sh
CREATE ROLE
CREATE DATABASE
GRANT

waiting for server to shut down...2019-04-06 11:16:40.287 UTC [43] LOG:  received fast shutdown request
.2019-04-06 11:16:40.290 UTC [43] LOG:  aborting any active transactions
2019-04-06 11:16:40.291 UTC [43] LOG:  background worker "logical replication launcher" (PID 50) exited with exit code 1
2019-04-06 11:16:40.293 UTC [45] LOG:  shutting down
2019-04-06 11:16:40.358 UTC [43] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

: not foundbin/docker-entrypoint.sh: line 176: exec:

Describe pod result:

Name:               idp-db-7d7d56b7cd-tz5bn
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               ip-10-2-3-179.ca-central-1.compute.internal/10.2.3.179
Start Time:         Sat, 06 Apr 2019 11:05:19 +0000
Labels:             app=idp-db
                    pod-template-hash=7d7d56b7cd
Annotations:        <none>
Status:             Running
IP:                 192.168.23.199
Controlled By:      ReplicaSet/idp-db-7d7d56b7cd
Containers:
  idp-db:
    Container ID:   docker://325889248dc0b80d5f2f0e8c945bb314563a35a767ef0b379b683c4d2b1a1dd9
    Image:          postgres
    Image ID:       docker-pullable://postgres@sha256:e002abfaf7b0f1015b7510f66b709e2bacbecc7762c428634c27a3c5e58ffbae
    Port:           5432/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    127
      Started:      Sat, 06 Apr 2019 11:16:38 +0000
      Finished:     Sat, 06 Apr 2019 11:16:40 +0000
    Ready:          False
    Restart Count:  7
    Environment:    <none>
    Mounts:
      /docker-entrypoint-initdb.d from init-script (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-ckwk8 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  init-script:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      idp-db-init-script
    Optional:  false
  db-data:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  default-token-ckwk8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-ckwk8
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From                                                  Message
  ----     ------     ----                  ----                                                  -------
  Normal   Scheduled  13m                   default-scheduler                                     Successfully assigned default/idp-db-7d7d56b7cd-tz5bn to ip-10-2-3-179.ca-central-1.compute.internal
  Normal   Created    12m (x4 over 13m)     kubelet, ip-10-2-3-179.ca-central-1.compute.internal  Created container
  Normal   Started    12m (x4 over 13m)     kubelet, ip-10-2-3-179.ca-central-1.compute.internal  Started container
  Normal   Pulling    11m (x5 over 13m)     kubelet, ip-10-2-3-179.ca-central-1.compute.internal  pulling image "postgres"
  Normal   Pulled     11m (x5 over 13m)     kubelet, ip-10-2-3-179.ca-central-1.compute.internal  Successfully pulled image "postgres"
  Warning  BackOff    3m21s (x47 over 13m)  kubelet, ip-10-2-3-179.ca-central-1.compute.internal  Back-off restarting failed container
-- Sudhakar
docker
kubernetes
postgresql

1 Answer

4/16/2019

As per @Sudhakar comment the issue has be resolved by recreating initial script, which was probably affected by some special character or another incorrect formatting case.

-- mk_sta
Source: StackOverflow