Grafana throws HTTP Error Bad Gateway for Prometheus data source

3/31/2019

I have Grafana and Prometheus set up on my k8s cluster. Both were installed thru helm using https://github.com/helm/charts/tree/master/stable. Both Grafana and Prometheus are set up thru k8s nginx ingress with my domian addresses. When I try to set up the Prometheus as a data source in Grafana I get HTTP Error Bad Gateway. In the chrome console in Grafana page I see:

http://grafana.domain.com/api/datasources/proxy/1/api/v1/query?query=1%2B1&time=1554043210.447

Grafana version: Grafana v6.0.0 (commit: 34a9a62)

Grafana data source settings for Prometheus: URL: https://prometheus.mydomain.com:9090

Access: Server(Default)

Auth: Basic & TLS Client Auth

What might be wrong and how to debug/fix it?

-- camel
grafana
kubernetes
kubernetes-helm
prometheus

1 Answer

3/31/2019

In Grafana data source settings for prometheus database add prometheus service dns and service port. Like below

<prometheus service name>. Namespace. Svc. Cluster. Local:9090
-- P Ekambaram
Source: StackOverflow