Is it possible to get a notification if kubernetes job fails

12/4/2018

I would like to know if it is possible to send notification using yaml config if the kubernetes job fails?

For example, I have a kubetnetes job which runs once in everyday. Now i have been running a jenkins job to check and send notification if the job fails. Do we have any options to get notification from kubernetes jobs directly if it fails? It should be something like we add in job yaml

-- Dinesh SC
jobs
kubernetes
yaml

1 Answer

12/4/2018

I'm not sure about any built in notification support. That seems like the kind of feature you can find in external dedicated monitoring/notification tools such as Prometheus or Logstash output.

For example, you can try this tutorial to leverage the prometheus metrics generated by default in many kubernetes clusters: https://medium.com/@tristan_96324/prometheus-k8s-cronjob-alerts-94bee7b90511

Or you can theoretically setup Logstash and monitor incoming logs sent by filebeat and conditionally send alerts as part of the output stage of the pipelines via the "email output plugin"

Other methods exist as well as mentioned in this similar issue: How to send alerts based on Kubernetes / Docker events?

For reference, you may also wish to read this request as discussed in github: https://github.com/kubernetes/kubernetes/issues/22207

-- yosefrow
Source: StackOverflow