Trying to solve dependency between pods using postStart lifecycle.
Use case: micro service A should start after the start of micro service B.
For that we have added one container (curl) which will check if dependent service is up or not using curl command.
But when we add any command in postStart lifecycle hook, pods keep restarting and goes in crashlookbackoff state
Deployment.yaml :
kind: Deployment
metadata:
name: Microservice-A-deployment
spec:
replicas: 1
selector:
matchLabels:
app: Microservice-A
template:
metadata:
labels:
app: Microservice-A
date: 20thJune2021
annotations:
sidecar.istio.io/rewriteAppHTTPProbers: "false"
proxy.istio.io/config: '{ "holdApplicationUntilProxyStarts": true }'
spec:
containers:
- name: curl
image: ewoutp/docker-nginx-curl
imagePullPolicy: IfNotPresent
command: [ 'sh', '-c', 'touch /tmp/healthy; echo The Pod is running && sleep 50' ]
livenessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 15
periodSeconds: 5
lifecycle:
postStart:
exec:
command: [ "/bin/sh", "-c", 'sleep 10;until [ $(eval curl -o -I -L -s -w "%{http_code}" http://microservice-B-api-service:9439/manage/health) -eq 200 ]; do echo "Waiting for microservice-B API";sleep 10; done; exit 0' ]
- name: Microservice-A
image: microserviceA:latest
imagePullPolicy: Always
ports:[![enter image description here][1]][1]
- name: port
containerPort: 8080
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 120
periodSeconds: 30
timeoutSeconds: 30
imagePullSecrets:
- name: dockersecret
Note: Reason for not using init-container: As we have implemented Istio with strict MTLS policy. https://github.com/istio/istio/issues/32039
That is because your command in postStart is sleeping for 10 seconds and your LivenessProbe
is configured to fail after 5 seconds.
Maybe increase initialDelaySeconds
or add a failureThreshold
.
You can use readinessProbe as well with the livenessProbe like this:
readinessProbe:
httpGet:
path: /api/health
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
failureThreshold: 8
// for tcpSocket use:
readinessProbe:
tcpSocket:
port: 3306