Unable to scrape Kubernetes nodes with Prometheus

7/20/2017

I am currently using Prometheus to scrape Kubernetes components. I am able to scrape pods only and not nodes. I followed the official documentation to write config file. I added "prometheus.io/scrape=true" annotation to the nodes. Still it is not working. What other notations should be there. This is the config file.

- job_name: "kubernetes-nodes"

# The URL scheme with which to fetch metrics from targets.
scheme: https

# Optional TLS configuration.
tls_config:
# Disable validation of the server certificate.
  insecure_skip_verify: true

kubernetes_sd_configs:
- api_servers:
  - https://kubernetes.default.svc
  role: node

relabel_configs:
- action: labelmap
  regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
  replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
  regex: (.+)
  target_label: __metrics_path__
  replacement: /api/v1/nodes/${1}/proxy/metrics

I went through the prometheus logs and this is the error-

level=error msg="Cannot initialize nodes collection: unable to list Kubernetes nodes: unable to query any API servers: Get https://kubernetes.default.svc/api/v1/nodes: x509: certificate signed by unknown authority" source="node.go:124"

I followed this example configuration file https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml

-- Shubham Kaushik
kubernetes
prometheus

0 Answers