how to run alertmanager at k8s branch cluster

2/17/2021

i have no idea how to run alertmanager.yml to integrate with opsgenie. what command should i use at terminal in order to make it run? i tried k create -f alertmanager.yml, but it shows: error: error validating "alertmanager.yml": error validating data: [apiVersion not set, kind not set]; if you choose to ignore these errors, turn validation off with --validate=false

here is my alertmanager.yml, any suggestion?

global:
  resolve_timeout: 1m
  opsgenie_api_url: https://api.opsgenie.com/
  opsgenie_api_key: <my_opsgenie_key>
receivers:
  - opsgenie_configs:
    - teams: test_escalation
      priority: '{{ range .Alerts }}{{ if eq .Labels.severity "critical"}}P1{{else if eq .Labels.severity "warning"}}P2{{else if eq .Labels.severity "info"}}P3{{else}}P4{{end}}{{end}}'
    name: opsgenie
route:
  group_by: ['...']
  receiver: opsgenie
  repeat_interval: 5m
-- Lawrence Liu
kubernetes
opsgenie
prometheus
prometheus-alertmanager

1 Answer

2/18/2021

Your alertmanager.yml file only contains the Alertmanager configuration that should be mounted to alertmanager Pod using ConfigMap or Secret (it depends on how did you deploy alertmanager).

I assume you already have Alertmanager running and just want to modify its configuration.
( If you don't have Alertmanager deployed, you can easily install it with e.g. helm: helm-alertmanager. )


I will describe how you can modify alertmanager configuration using an example.

First you need to find a ConfigMap where the Alertmanager configuration is located.

### kubectl describe pod <ALERTMANAGER POD NAME>

$ kubectl describe pod alertmanager-0
Name:         alertmanager-0
Namespace:    default
...
Containers:
  alertmanager:
    ...
    Mounts:
      /etc/alertmanager from config (rw)
...
Volumes:
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      alertmanager
    Optional:  false
...

In the example above we can see that the configuration is in ConfigMap named alertmanager.

You can edit this Configmap:

### kubectl edit cm <CONFIGMAP NAME>

$ kubectl edit cm alertmanager

or you can replace it using your alertmanager.yml file:

### kubectl delete cm <CONFIGMAP NAME> && kubectl create cm <CONFIGMAP NAME> --from-file alertmanager.yml

$ kubectl delete cm alertmanager && kubectl create cm alertmanager --from-file alertmanager.yml

Additionally, I don't know which version of Alertmanager you are using, but is seems that from v0.16.1 we can use responders field instead of teams. You can find more information here.

-- matt_j
Source: StackOverflow