I'm following the steps here to deploy a multi container application in Kubernetes https://stackoverflow.com/questions/70637470/kubernetes-deployment-not-reachable-via-browser-exposed-with-service
When I browse to nodeIP:nodePort, the application says "database connection mysql is missing or could not be created". I suspect the db is not talking to the app pod.
I've been able to exec into the db using the credentials provisioned in the app ENV. The database.php file also has the correct credentials in the app pod.
DB pod logs
root@osboxes:/home/osboxes# kubectl logs eramba-mariadb-0 -n eramba-1
mariadb 03:25:18.67
mariadb 03:25:18.67 Welcome to the Bitnami mariadb container
mariadb 03:25:18.67 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mariadb
mariadb 03:25:18.68 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mariadb/issues
mariadb 03:25:18.68
mariadb 03:25:18.68 INFO ==> ** Starting MariaDB setup **
mariadb 03:25:18.70 INFO ==> Validating settings in MYSQL_*/MARIADB_* env vars
mariadb 03:25:18.71 INFO ==> Initializing mariadb database
mariadb 03:25:18.73 WARN ==> The mariadb configuration file '/opt/bitnami/mariadb/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file.
mariadb 03:25:18.73 INFO ==> Using persisted data
mariadb 03:25:18.78 INFO ==> Running mysql_upgrade
mariadb 03:25:18.79 INFO ==> Starting mariadb in background
mariadb 03:25:20.87 INFO ==> Stopping mariadb
mariadb 03:25:21.89 INFO ==> ** MariaDB setup finished! **
mariadb 03:25:21.95 INFO ==> ** Starting MariaDB **
2022-01-10 3:25:21 0 [Note] /opt/bitnami/mariadb/sbin/mysqld (mysqld 10.5.13-MariaDB) starting as process 1 ...
2022-01-10 3:25:22 0 [Note] InnoDB: Uses event mutexes
2022-01-10 3:25:22 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2022-01-10 3:25:22 0 [Note] InnoDB: Number of pools: 1
2022-01-10 3:25:22 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2022-01-10 3:25:22 0 [Note] mysqld: O_TMPFILE is not supported on /opt/bitnami/mariadb/tmp (disabling future attempts)
2022-01-10 3:25:22 0 [Note] InnoDB: Using Linux native AIO
2022-01-10 3:25:22 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2022-01-10 3:25:22 0 [Note] InnoDB: Completed initialization of buffer pool
2022-01-10 3:25:22 0 [Note] InnoDB: 128 rollback segments are active.
2022-01-10 3:25:22 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-01-10 3:25:22 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-01-10 3:25:22 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-01-10 3:25:22 0 [Note] InnoDB: 10.5.13 started; log sequence number 3829693; transaction id 2899
2022-01-10 3:25:22 0 [Note] Plugin 'FEEDBACK' is disabled.
2022-01-10 3:25:22 0 [Note] InnoDB: Loading buffer pool(s) from /bitnami/mariadb/data/ib_buffer_pool
2022-01-10 3:25:22 0 [Note] Server socket created on IP: '::'.
2022-01-10 3:25:22 0 [Warning] 'proxies_priv' entry '@% root@eramba-mariadb-0' ignored in --skip-name-resolve mode.
2022-01-10 3:25:22 0 [Note] Reading of all Master_info entries succeeded
2022-01-10 3:25:22 0 [Note] Added new Master_info '' to hash table
2022-01-10 3:25:22 0 [Note] /opt/bitnami/mariadb/sbin/mysqld: ready for connections.
Version: '10.5.13-MariaDB' socket: '/opt/bitnami/mariadb/tmp/mysql.sock' port: 3306 Source distribution
2022-01-10 3:25:22 0 [Note] InnoDB: Buffer pool(s) load completed at 220110 3:25:2
describe for db service
root@osboxes:/home/osboxes# kubectl describe svc eramba-mariadb -n eramba-1
Name: eramba-mariadb
Namespace: eramba-1
Labels: app.kubernetes.io/component=primary
app.kubernetes.io/instance=eramba
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=mariadb
helm.sh/chart=mariadb-10.3.0
Annotations: meta.helm.sh/release-name: eramba
meta.helm.sh/release-namespace: eramba-1
Selector: app.kubernetes.io/component=primary,app.kubernetes.io/instance=eramba,app.kubernetes.io/name=mariadb
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.107.247.50
IPs: 10.107.247.50
Port: mysql 3306/TCP
TargetPort: mysql/TCP
Endpoints: 10.20.0.28:3306
Session Affinity: None
Events: <none>
describe db pod
root@osboxes:/home/osboxes# kubectl describe pod eramba-mariadb-0 -n eramba-1
Name: eramba-mariadb-0
Namespace: eramba-1
Priority: 0
Node: osboxes/172.16.42.135
Start Time: Sun, 09 Jan 2022 20:59:24 -0500
Labels: app.kubernetes.io/component=primary
app.kubernetes.io/instance=eramba
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=mariadb
controller-revision-hash=eramba-mariadb-64579c97cc
helm.sh/chart=mariadb-10.3.0
statefulset.kubernetes.io/pod-name=eramba-mariadb-0
Annotations: checksum/configuration: dcf82b6bca1415cc418030332ecbc15240b060c806b11c7c0319fa2fa568ca69
Status: Running
IP: 10.20.0.28
IPs:
IP: 10.20.0.28
Controlled By: StatefulSet/eramba-mariadb
Init Containers:
volume-permissions:
Container ID: docker://81e4c5f3b6cfc2d638cb366a4c64b22088e3a205fffd3ef15f4eafac804690f6
Image: docker.io/bitnami/bitnami-shell:10-debian-10-r279
Image ID: docker-pullable://bitnami/bitnami-shell@sha256:b9dba2d6bf011513a8914bdbd2c3307c9253d235d2c50d158b5d1c1359a6fe39
Port: <none>
Host Port: <none>
Command:
/bin/bash
-ec
chown -R 1001:1001 /bitnami/mariadb
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 09 Jan 2022 22:25:17 -0500
Finished: Sun, 09 Jan 2022 22:25:18 -0500
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/bitnami/mariadb from data (rw)
Containers:
mariadb:
Container ID: docker://ece671c8646d9c12c5a56b490145fe8ac8de223f2c5ca18a89041cdce92bbe13
Image: docker.io/bitnami/mariadb:10.5.13-debian-10-r32
Image ID: docker-pullable://bitnami/mariadb@sha256:4969eda3a6cbb8007b4e52992979d9e1f1685cbed7c21afd2b44a64797c9e400
Port: 3306/TCP
Host Port: 0/TCP
State: Running
Started: Sun, 09 Jan 2022 22:25:18 -0500
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 09 Jan 2022 20:59:26 -0500
Finished: Sun, 09 Jan 2022 21:59:43 -0500
Ready: True
Restart Count: 1
Liveness: exec [/bin/bash -ec password_aux="${MARIADB_ROOT_PASSWORD:-}"
if [[ -f "${MARIADB_ROOT_PASSWORD_FILE:-}" ]]; then
password_aux=$(cat "$MARIADB_ROOT_PASSWORD_FILE")
fi
mysqladmin status -uroot -p"${password_aux}"
] delay=120s timeout=1s period=10s #success=1 #failure=3
Readiness: exec [/bin/bash -ec password_aux="${MARIADB_ROOT_PASSWORD:-}"
if [[ -f "${MARIADB_ROOT_PASSWORD_FILE:-}" ]]; then
password_aux=$(cat "$MARIADB_ROOT_PASSWORD_FILE")
fi
mysqladmin status -uroot -p"${password_aux}"
] delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
BITNAMI_DEBUG: false
MARIADB_ROOT_PASSWORD: <set to the key 'mariadb-root-password' in secret 'eramba-mariadb'> Optional: false
MARIADB_DATABASE: erambadb
Mounts:
/bitnami/mariadb from data (rw)
/docker-entrypoint-initdb.d from custom-init-scripts (rw)
/opt/bitnami/mariadb/conf/my.cnf from config (rw,path="my.cnf")
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: eramba-mariadb
Optional: false
custom-init-scripts:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: eramba
Optional: false
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: eramba-storage
ReadOnly: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 93m default-scheduler Successfully assigned eramba-1/eramba-mariadb-0 to osboxes
Normal Pulled 93m kubelet Container image "docker.io/bitnami/bitnami-shell:10-debian-10-r279" already present on machine
Normal Created 93m kubelet Created container volume-permissions
Normal Started 93m kubelet Started container volume-permissions
Normal Pulled 93m kubelet Container image "docker.io/bitnami/mariadb:10.5.13-debian-10-r32" already present on machine
Normal Created 93m kubelet Created container mariadb
Normal Started 93m kubelet Started container mariadb
Normal SandboxChanged 7m52s kubelet Pod sandbox changed, it will be killed and re-created.
Warning Failed 7m51s (x2 over 7m52s) kubelet Error: services have not yet been read at least once, cannot construct envvars
Normal Pulled 7m49s (x3 over 7m52s) kubelet Container image "docker.io/bitnami/bitnami-shell:10-debian-10-r279" already present on machine
Normal Created 7m49s kubelet Created container volume-permissions
Normal Started 7m49s kubelet Started container volume-permissions
Normal Pulled 7m48s kubelet Container image "docker.io/bitnami/mariadb:10.5.13-debian-10-r32" already present on machine
Normal Created 7m48s kubelet Created container mariadb
Normal Started 7m48s kubelet Started container mariadb
root@osboxes:/home/osboxes#
Logs for the web-app pod
root@osboxes:/home/osboxes# kubectl logs eramba-web-5d7ffc87fd-lsrck -n eramba-1
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.20.0.30. Set the 'ServerName' directive globally to suppress this message
describe web service
root@osboxes:/home/osboxes# kubectl describe svc eramba-web -n eramba-1
Name: eramba-web
Namespace: eramba-1
Labels: app.kubernetes.io/name=eramba-web
Annotations: <none>
Selector: app.kubernetes.io/name=eramba-web
Type: NodePort
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.100.24.86
IPs: 10.100.24.86
Port: http 8080/TCP
TargetPort: 8080/TCP
NodePort: http 31622/TCP
Endpoints: 10.20.0.30:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
describe web-app pod
root@osboxes:/home/osboxes# kubectl describe pod eramba-web-5d7ffc87fd-lsrck -n eramba-1
Name: eramba-web-5d7ffc87fd-lsrck
Namespace: eramba-1
Priority: 0
Node: osboxes/172.16.42.135
Start Time: Sun, 09 Jan 2022 21:17:47 -0500
Labels: app.kubernetes.io/name=eramba-web
pod-template-hash=5d7ffc87fd
Annotations: <none>
Status: Running
IP: 10.20.0.30
IPs:
IP: 10.20.0.30
Controlled By: ReplicaSet/eramba-web-5d7ffc87fd
Containers:
eramba-web:
Container ID: docker://e76bcea65ab2f5e50877e3a5504ec515024a1728fcd31465588605dc2fe74a35
Image: markz0r/eramba-app:c281
Image ID: docker-pullable://markz0r/eramba-app@sha256:747f89930be922c7a6ca51a140890d22f2c54de74fa391e421b074fb14037198
Port: 8080/TCP
Host Port: 0/TCP
State: Running
Started: Sun, 09 Jan 2022 22:25:16 -0500
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 09 Jan 2022 21:17:48 -0500
Finished: Sun, 09 Jan 2022 21:59:43 -0500
Ready: True
Restart Count: 1
Environment:
MYSQL_HOSTNAME: eramba-mariadb
MYSQL_DATABASE: erambadb
MYSQL_USER: root
MYSQL_PASSWORD: eramba
DATABASE_PREFIX:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bzjp6 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-bzjp6:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 80m default-scheduler Successfully assigned eramba-1/eramba-web-5d7ffc87fd-lsrck to osboxes
Normal Pulled 80m kubelet Container image "markz0r/eramba-app:c281" already present on machine
Normal Created 80m kubelet Created container eramba-web
Normal Started 80m kubelet Started container eramba-web
Normal SandboxChanged 12m kubelet Pod sandbox changed, it will be killed and re-created.
Warning Failed 12m kubelet Error: services have not yet been read at least once, cannot construct envvars
Normal Pulled 12m (x2 over 12m) kubelet Container image "markz0r/eramba-app:c281" already present on machine
Normal Created 12m kubelet Created container eramba-web
Normal Started 12m kubelet Started container eramba-web
output
root@osboxes:/home/osboxes# kubectl get cm,pvc,pv,svc,pods -n eramba-1
NAME DATA AGE
configmap/eramba 1 91m
configmap/eramba-mariadb 1 90m
configmap/kube-root-ca.crt 1 91m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/eramba-storage Bound eramba-storage 5Gi RWO eramba-storage 91m
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/eramba-storage 5Gi RWO Retain Bound eramba-1/eramba-storage eramba-storage 91m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/eramba-mariadb ClusterIP 10.107.247.50 <none> 3306/TCP 90m
service/eramba-web NodePort 10.100.24.86 <none> 8080:31622/TCP 72m
NAME READY STATUS RESTARTS AGE
pod/eramba-mariadb-0 1/1 Running 1 (30m ago) 90m
pod/eramba-web-5d7ffc87fd-lsrck 1/1 Running 1 (30m ago) 72m