I am running node exporter on GKE on a different port (11100) and configuring prometheus.yml to use kubernetes_sd_configs. However, the service discovery seems to be returning the node IP with kublet port (10250) <node-ip>:10250/metrics. I can’t seem to find a way to specify which port to use. Any ideas?

 - job_name: gke-nodes
      kubernetes_sd_configs:
      - role: node

Also, metric exporter started on the correct port I validated by running curl in the internal node IP <node-ip>:11100/metrics and it works like a charm

Here is my Node exporter definition

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter-ds
  namespace: monitoring
  labels:
    app: node-exporter
    belongsTo: monitoring
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      serviceAccountName: monitoring-sa
      volumes:
        - name: proc
          hostPath:
            path: /proc
        - name: sys
          hostPath:
            path: /sys
      containers:
        - name: node-exporter
          image: prom/node-exporter:v0.18.1
          args:
            - "--web.listen-address=0.0.0.0:11100"
            - "--path.procfs=/proc_host"
            - "--path.sysfs=/host_sys"
          ports:
            - containerPort: 11100
              hostPort: 11100
          volumeMounts:
          - name: sys
            readOnly: true
            mountPath: /host_sys
          - name: proc
            readOnly: true
            mountPath: /proc_host
          imagePullPolicy: IfNotPresent
      hostNetwork: true
      hostPID: true