I want to setup statsd-exporter
as DaemonSet on my Kubernetes cluster. It exposes a UDP port 9125, on which applications can send metrics using statsD client library. Prometheus
crawlers can crawl this exporter for application or system metrics. I want to send the metrics to the UDP server running in the exporter on port 9125. I have two options:
Expose a service as ClusterIP
for the DaemonSet and then configure the statsD clients to use that IP and Port for sending metrics
Make the statsd-exporter
run on hostNetwork
, and somehow enable the pods to send metrics to exporter
running on the same node.
Somehow, option 2 seems better, since my pods will be sending metrics to an exporter running on the same node, but I am not able to send metrics to the local pod of statsd-exporter
since I don't have the IP of the node the pod is running on.
Can you please compare the pros and cons of both methods, and suggest how can I know the IP address of Node on which the pod is running along with the exporter.
EDIT 1
I am able to get the Node IP by adding the environment variable.
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
I still need clarity on which will be better approach to setting this up. Exposing a service with type ClusterIP
and then using the HOST:PORT
from environment variable in the pod or use hostNetwork: true
in pod spec and then access it using NODE_IP
environment variable. Does clusterIp guarantees that the packet will be routed to the same node as the pod sending the packet?
EDIT 2
I explored headless service and I think this comes closest to what I want. But I am not sure that the DNS resolution will return the local node IP as the first result in nslookup
.
I would suggest one of the below two approaches, both have their pros and cons.
Daemon set using hostPort
. It would be fast as both pods would be on the same node, but the statsd port would be exposed. (You need to secure statsd by some other way)
hostPort
but secureExposing a service and using the service dns to connect (servicename.namespace.svc.cluster.local
). Not as fast as the hostPort
as there is no way to reach specific pod, but secure as no one from outside the cluster can hit the statsd.
More details: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/#communicating-with-daemon-pods