K
Q

Prometheus dashboard exposed over ingress controller

12/30/2019

I am trying to setup Prometheus in k8 cluster, able to run using helm. Accessing dashboard when i expose prometheus-server as LoadBalancer service using external ip. Same does not work when I try to configure this service as ClusterIP and making it as backend using ingress controller. Receiving 404 error, any thoughts on how to troubleshoot this?

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ops-ingress
  annotations:
    #nginx.org/server-snippet: "proxy_ssl_verify off;"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
      - path: /prometheus(/|$)(.*)
        backend:
          serviceName: prometheus-server
          servicePort: 80

with above ingress definition in place, url “http://<>/prometheus/ getting redirected to http://<>/graph/ and then 404 error page getting rendered. When url adjusted to http://<>/prometheus/graph some of webcontrols gets rendered with lots of errors on browser console.

-- Revanth
kubernetes
kubernetes-ingress
nginx-ingress
prometheus
prometheus-operator

Similar Questions

Filebeat and json logs from Kubernetes not working
traefik kubernetes crd health check
Completely remove Kubernetes
How to verify cluster network policy configuration/support

2 Answers

1/2/2020

Please change your Ingress configuration file, add host field:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ops-ingress
  annotations:
    #nginx.org/server-snippet: "proxy_ssl_verify off;"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: prometheus.example.com
    http:
      paths:
      - path: /prometheus(/|$)(.*)
        backend:
          serviceName: prometheus-server
          servicePort: 80

then apply changes executing command:

$ kubectl aply -f your_ingress_congifguration_file.yaml

The host header field in a request provides the host and port information from the target URI, enabling the origin server to distinguish among resources while servicing requests for multiple host names on a single IP address.

Please take a look here: hosts-header.

Ingress definition: ingress.

Useful information: helm-prometheus.

Useful documentation: ingress-path-matching.

-- MaggieO
Source: StackOverflow
1/1/2020

Prometheus might be expecting to have control over the root path (/).

Please change the Ingress to prometheus.example.com and it should work fine. (Changing it to a subdomain)

-- Tummala Dhanvi
Source: StackOverflow