How to install helm chart prometheus-operator on GKE with prometheusOperator.admissionWebhooks.enabled=false?

11/22/2019

I want to install the helm chart stable/prometheus-operator on a GKE cluster. I'm aware that either firewall rules need to be adjusted or hooks need to be disabled by setting prometheusOperator.admissionWebhooks.enabled=false (for details see the README of the chart).

However, if I install the chart with

    - wget -qq https://get.helm.sh/helm-v3.0.0-linux-amd64.tar.gz && tar xf helm-v3.0.0-linux-amd64.tar.gz && mv linux-amd64/helm /usr/local/bin
    - helm repo add stable https://kubernetes-charts.storage.googleapis.com/
    - helm repo update
    - kubectl create ns monitoring
    - kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/master/example/prometheus-operator-crd/alertmanager.crd.yaml
    - kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/master/example/prometheus-operator-crd/prometheus.crd.yaml
    - kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/master/example/prometheus-operator-crd/prometheusrule.crd.yaml
    - kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/master/example/prometheus-operator-crd/servicemonitor.crd.yaml
    - kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/master/example/prometheus-operator-crd/podmonitor.crd.yaml
    - helm install monitoring stable/prometheus-operator --namespace=monitoring --wait --timeout 10m --set prometheusOperator.admissionWebhooks.enabled=false

in GitLab CI the pod prometheus-operator has two containers which remain in state "Pending" for 5 minutes. I expect this rather simple setup to be available within one minute.

You can inspect the cluster setup at https://gitlab.com/krichter/prometheus-operator-503/-/jobs/358887366.

The approach shown in Installing Prometheus on GKE + istio doesn't apply because I didn't install istio.

-- Karl Richter
google-kubernetes-engine
kubernetes
kubernetes-helm
prometheus
prometheus-operator

1 Answer

12/2/2019

This is caused by a known issue in the helm chart. According to https://github.com/helm/charts/issues/19147 the issue can be avoided by setting prometheusOperator.tlsProxy.enabled=false.

-- Karl Richter
Source: StackOverflow