K
Q

Prometheus operator AlertmanagerConfig unknown field "webhook_configs"

3/1/2021

After applying these manifests https://github.com/prometheus-operator/kube-prometheus/blob/main/kustomization.yaml I want to create AlertManager webhook:

apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
  name: custom
spec:
  route:
    receiver: custom
    groupBy: ['job']
    groupWait: 30s
    groupInterval: 5m
    repeatInterval: 12h
  receivers:
    - name: custom
      webhook_configs:
        - send_resolved: true
          url: https://example.com

getting an error:

error: error validating "alertmanagerconfig.yaml": error validating data: ValidationError(AlertmanagerConfig.spec.receivers[0]): unknown field "webhook_configs" in com.coreos.monitoring.v1alpha1.AlertmanagerConfig.spec.receivers; if you choose to ignore these errors, turn validation off with --validate=false

How to fix that?

-- Jonas
kubernetes
prometheus-alertmanager
prometheus-operator

Similar Questions

Expose services locally
What is the "port" used for for a Kubernetes Service
How to install kubernetes on multiple machines simultaneously?
External DNS + SRV
Rancher - New to Existing Cluster

4 Answers

11/13/2021

In addition to the answer from Kiran (Thanks!) one small correction - it is actually sendResolved and not send_resolved. 

apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
  name: custom
spec:
  receivers:
    - name: custom
      webhookConfigs:
        - sendResolved: true
          url: https://example.com
-- buLLfisH
Source: StackOverflow
11/15/2021

I solved this issue by generating Kubernetes secret named alertmanager-main using this yaml:

global:
  #skipped
route:
  group_by: ['job']
  receiver: Default
  routes:
    - receiver: Watchdog
      repeat_interval: 1m
      match:
        alertname: Watchdog  

receivers:
  - name: Watchdog
    webhook_configs:
      - url: https://my.watchdog.webhook.url

  # Other things

This is Alert Manager config.

I am answering my question after some time and probably this issue with the Prometheus Operator CRD is fixed by now.

-- Jonas
Source: StackOverflow
3/1/2021

Taking this link as reference,

apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
  name: custom
spec:
  receivers:
    - name: custom
      webhookConfigs:
        - send_resolved: true
          url: https://example.com

You can also check field using kubectl explain command

kubectl explain alertmanagerconfig.spec.receivers

-- Kiran
Source: StackOverflow
5/23/2021

The problem is that you are using the configuration reference for the actual AlertManager application and not the one for the Kubernetes Custom Resource provided by the Prometheus Operator.

The Prometheus Operator takes the configuration provided in the form of custom resources like AlertManagerConfig and converts them into actual AlertManager config and updates the configuration file the application uses. That is a part of the reason why you use an operator in the first place. It makes these things convenient for you.

So the actual configuration reference you should be using here is this. This part of the Prometheus Operator's github readme lists down the available custom resources you can use with it.

-- aaruja
Source: StackOverflow