Is there an ability to deploy Prometheus Operator on a distributed environment?

6/25/2019

What I have: Prometheus Operator Helm chart deployed on a single Kubernetes cluster the same which is also application cluster. So, all of the application-related pods, prometheus exporter pods, Grafana pod and Prometheus itself live within the same cluster.

What I want: I want to split the above configuration is such a way, that I have all the application-related as well as exporter pods exist in application cluster, while having Prometheus and Grafana pods deployed to separate one(in order to not have single point of failure)

Is there a way to achieve this using Prometheus Operator?

-- gokareless
kubernetes
kubernetes-helm
monitoring
prometheus-operator

2 Answers

6/25/2019

Technically, maybe, kind of. Using kubefed you can sync object state between clusters. You would need to have a mutually flat network so that the Prom pod in one cluster can talk directly to the target pods in the other.

Overall this is probably not a good idea though. Set up prom in each application cluster and then use prom-to-prom federation instead to have a single overall install.

-- coderanger
Source: StackOverflow

6/25/2019

It's not quite what you're asking, but you could run these workloads in the same cluster, just on different nodes using node selectors or taints.

It would keep the monitoring/observability workloads separate from the application workloads without having to re-architect the cluster's network or deal with federation.

-- switchboard.op
Source: StackOverflow