External dependency in Pod Readiness and Liveness

11/22/2021

I am new for pod health check with Readiness and Liveness. Recently I am working on Readiness. The scenario is as following:

The pod is a RestAPI service, it needs to connect to Database and store information in DB. So if RestAPI service wants to offer service, it needs to make sure database connection is successfully.

Si in our pod Readiness logic implementation, we use HTTP-Get and check if DB connection is connected, if it is okay, then HTTP-Get returns Ok, otherwise Readiness will be failed.

Not sure if the above logic is reasonable? Or is there any other approach for this logic processing?

Apart from Readiness, how about Liveness? Do I need to check DB connection in order to check Liveness is okay?

Any idea and suggestion are appreciated

-- Joe
kubernetes
openshift

1 Answer

11/22/2021

readiness and liveness is mostly for service you are running inside container, there could be a scenario where your DB is up but there is issue with the application at that time also your readiness will be Up as DB is running, in ideal scenario if application not working it should stop accepting traffic.

i would recommend using the Init container or Life cycle hook for checking the condition of the Database first if it's up process will move ahead and your application or deployment will come into the picture.

If the application works well your readiness and liveness will HTTP-OK and the service start accepting traffic.

init container example

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
  initContainers:
  - name: init-myservice
    image: busybox
    command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;']
  - name: init-mydb
    image: busybox
    command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;']

Extra Notes

There is actually no need to check the DB readiness at all.

As your application will be trying to connect with the Database so if DB is not UP your application won't respond HTTP-OK so your application won't start, and readiness keep failing for application.

As soon as your Database comes up your application will create a successful connection with DB and it will give 200 responses and readiness will mark POD ready.

there is no extra requirement to setup the readiness for Db and based on that start POD.

-- Harsh Manvar
Source: StackOverflow