I am trying to monitor my cluster Kubernetes and I am using prometheus to get all informations.
It's working perfectly but, I need to monitor some specific workers and I need to label it using __meta_kubernetes_namespace, but could not find any reference explaining how to change it in the kubnernetes Environment.
Please help me to resolve it,
Thank you.
You would need to write your own scrape config for Prometheus.
This is a partial snipit of istio envoy proxies stats, you should write your own based on what resources to monitor and where to look for them:
- job_name: 'envoy-stats'
metrics_path: /stats/prometheus
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_container_port_name]
action: keep
regex: '.*-envoy-prom'
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: pod_name
metric_relabel_configs:
# Exclude some of the envoy metrics that have massive cardinality
# This list may need to be pruned further moving forward, as informed
# by performance and scalability testing.
- source_labels: [ __name__ ]
regex: 'envoy_http_(stats|admin).*'
action: drop
- source_labels: [ __name__ ]
regex: 'envoy_cluster_(lb|retry|bind|internal|max|original).*'
action: dropAlso please go over first steps with prometheus as this will be helpful in working with Prometheus.