I have a cronjob that runs and does things regularly. I want to send a slack message with the technosophos/slack-notify container when that cronjob fails.
Is it possible to have a container run when a pod fails?
There is nothing built in for this that i am aware of. You could use a web hook to get notified when a pod changes and look for state stuff in there. But you would have to build the plumbing yourself or look for an existing third party tool.
Pods and Jobs are different things. If you want to wait for a job that has failed and send an email after it has, you can do something like this in bash:
while true
do
kubectl wait --for=condition=failed job/myjob
kubectl run --image=technosophos/slack-notify --env="EMAIL=failure@yourdomain.com"
done
To the question: Is it possible to have a container run when a pod fails?
Yes , although there is nothing out of the box right now , but you can define a health check.
Then you can write a cron job , or a Jenkins job , or a custom kubernetes cluster service/controller that checks/probes that health check regularly and if the health check fails then you can run a container based on that.