Alert for restarting and terminating pods

2/12/2020

I need to apply some alerts using Prometheus rules to get to know which pods were restarted also which pods were terminated. I have found some rules specific for restarting and terminating pods but i had some clarification regarding that.

For example:

Example group with one alert

groups:
- name: example-alert
  rules:
    # Alert about restarts
  - alert: RestartAlerts
    expr: count(kube_pod_container_status_restarts_total) by (pod-name) > 5
    for: 10m
    annotations:
      summary: "More than 5 restarts in pod {{ $labels.pod-name }}"
      description: "{{ $labels.container-name }} restarted (current value: {{ $value }}s) times in pod {{ $labels.pod-namespace }}/{{ $labels.pod-name }}"

so, in this case, how do we know the pod-name,container-name .both of these can be updated for every new deployment, For this example how does it will work??

-- Riby Varghese
containers
google-kubernetes-engine
kubernetes
kubernetes-pod
prometheus

0 Answers