My idea was to implement a liveness probe as a command, and use something like
$ grep something ERROR
from inside a pod, so that if in the output of a pod, a line containing ERROR exists, the liveness probe fails.
Is this possible? If not, is it possible if I add another container in the same pod, to monitor the first container?
You could query the Kubernetes API server.
The request looks like this:
GET /api/v1/namespaces/{namespace}/pods/{name}/log
To use the token that's usually mounted in a Pod, you can call it like this:
curl https://kubernetes/api/v1/namespaces/default/pods/$HOSTNAME/log -k \
-H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
Sure you can have more then one container per pod. It's just not a standard approach because it violate the "one process per container" principle. You can read this article explaining multi-container pods in Kubernetes, what are the use cases and so on.
apiVersion: v1
kind: Pod
metadata:
name: test
spec:
containers:
- name: 1st
image: nginx
volumeMounts:
- name: html
mountPath: /tmp/html
livenessProbe:
exec:
command:
- cat
- /tmp/html/test
initialDelaySeconds: 10
periodSeconds: 3
- name: 2nd
image: debian
volumeMounts:
- name: html
mountPath: /tmp/html
command: ["/bin/sh", "-c"]
args:
- while true; do
date >> /tmp/html/test;
sleep 5;
done
volumes:
- name: html
emptyDir: {}
In this example pod
have two containers. Container 1st
running nginx
with mounted /tmp/html
and livenessProbe
which after 10 seconds from start checks if file /tmp/html/test
exists every 3 seconds and if it's missing it restarts the container. And 2nd
container running debian
with /tmp/html/
mounted, but it is also adding data entry into file /tmp/html/test
each 5 seconds.
In the above example if you remove the file manually and probe caches it it will restart 1st
container.
You would have to tailor this example into your particular needs, for example use grep -q ERROR /tml/html/test
which if succeed removes the /tmp/html/test
or change the probe itself.