We have a prometheus server + grafana running on bare metal (A).

We are also using prometheus federation to forward our stats from our kubernetes cluster (running on azure) to our bare metal prometheus server (B).

Using the bare-metal server (A) i can see that the metrics from (B) are coming through, however when using the datasource prometheus with a few kubernetes dashboards, i am unable to get any metrics on grafana.

I have used a combination of the following resources to set this up, but am unable to get the metrics in grafana dashboards, can anyone help me on this?

https://devopscube.com/install-configure-prometheus-linux/

https://grafana.com/docs/grafana/latest/installation/debian/

https://medium.com/@jotak/prometheus-federation-in-kubernetes-4ce46bda834e