According to Prometheus documentation:
"If a container has no specified ports, a port-free target per container is created for manually adding a port via relabeling."
However, it seems in my setup, pods with containerports have their address label converted as expected, but those without containerports are left "portless". Is this common/accepted behavior, or is there something I should change in order to apply __meta_kubernetes_pod_annotation_prometheus_io_port as the address port? Thanks!
additionalScrapeConfigs:
- job_name: 'perpod-metricsscrape'
scrape_interval: 5s
metrics_path: /metrics
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [ __meta_kubernetes_pod_annotation_prometheus_io_scrape ]
action: keep
regex: true
- source_labels: [ __meta_kubernetes_pod_annotation_prometheus_io_path ]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [ __address__, __meta_kubernetes_pod_annotation_prometheus_io_port ]
action: replace
regex: (.+)(?::\d+);(\d+)
replacement: ${1}:${2}
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
BTW, I altered the regex slightly from most examples floating around, thinking they weren't matching an address without the port (is as is the case with the aforementioned containerport-less pods) - before relabeling: _ address _="1.2.3.4"
Found the issue in the regex, and replacement... I now match port-less _ address , capture optional port, and replace with _ meta_kubernetes_pod_annotation_prometheus_io_port:
regex: ([^:]+)(:\d+)?;(\d+)
replacement: ${1}:${3}