K
Q

Kuberntes/Prometheus - Unable to Annotate in service file

5/24/2017

My Kubernetes version is :

# kubectl --version
Kubernetes v1.4.0

I am planning to use Prometheus to monitor my Kube cluster. For this, I need to annotate the metrics URL.

My current metrics URL is like :

http://172.16.33.7:8080/metrics

But I want it to be like :

http://172.16.33.7:8080/websocket/metrics

First I tried to do this manually ::

kubectl annotate pods websocket-backend-controller-db83999c5b534b277b82badf6c152cb9m1 prometheus.io/path=/websocket/metrics
kubectl annotate pods websocket-backend-controller-db83999c5b534b277b82badf6c152cb9m1 prometheus.io/scrape='true'
kubectl annotate pods websocket-backend-controller-db83999c5b534b277b82badf6c152cb9m1 prometheus.io/port='8080'

All these commands work perfectly fine and I am able to see the annotations.

{
  "metadata": {
    "name": "websocket-backend-controller-v1krf",
    "generateName": "websocket-backend-controller-",
    "namespace": "default",
    "selfLink": "/api/v1/namespaces/default/pods/websocket-backend-controller-v1krf",
    "uid": "e323994b-4081-11e7-8bd0-0050569b6f44",
    "resourceVersion": "27534379",
    "creationTimestamp": "2017-05-24T13:07:06Z",
    "labels": {
      "name": "websocket-backend"
    },
    "annotations": {
      "kubernetes.io/created-by": "{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"websocket-backend-controller\",\"uid\":\"e321f1a8-4081-11e7-8bd0-0050569b6f44\",\"apiVersion\":\"v1\",\"resourceVersion\":\"27531840\"}}\n",
      "prometheus.io/path": "/websocket/metrics",
      "prometheus.io/port": "8080",
      "prometheus.io/scrape": "true"
    }

But since I want this configuration to remain permanent, I am setting the following annotations in my services files.

# cat websocket-service.yaml 
apiVersion: v1
kind: Service
metadata:
  name: websocket-service
  labels:
    baseApi: websocket
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/path: /websocket/metrics
    prometheus.io/port: '8080'
spec:
  selector:
    name: websocket-backend
  ports:
    - port: 8080
      targetPort: 8080
      nodePort: 30800
      protocol: TCP
  type: NodePort
  clusterIP: 10.100.10.45

I restarted my websocket service and the corresponding pods but these configs do not seem to be taking effect.

kubectl create -f websocket-service.yaml
kubectl create -f ../controllers/websocket-replication-controller.yaml

The result does not show the annotations configured.

{
      "metadata": {
        "name": "websocket-backend-controller-v1krf",
        "generateName": "websocket-backend-controller-",
        "namespace": "default",
        "selfLink": "/api/v1/namespaces/default/pods/websocket-backend-controller-v1krf",
        "uid": "e323994b-4081-11e7-8bd0-0050569b6f44",
        "resourceVersion": "27531879",
        "creationTimestamp": "2017-05-24T13:07:06Z",
        "labels": {
          "name": "websocket-backend"
        },
        "annotations": {
          "kubernetes.io/created-by": "{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"websocket-backend-controller\",\"uid\":\"e321f1a8-4081-11e7-8bd0-0050569b6f44\",\"apiVersion\":\"v1\",\"resourceVersion\":\"27531840\"}}\n"
        }

All I am doing is rather than using a command line, I am setting the configs using services config but it does not seem to be working.

--
annotations
kubernetes
prometheus

Similar Questions

Tomcat deployment using Docker and Kubernetes
Unable to pull from private docker hub registry on kubernetes
Piping using toolbox in Google Container Optimized OS
Datadog count metric dropping with multiple containers

1 Answer

5/24/2017

If you annotate the service, it doesn't take any effect on the possibly matched pods. Your pods are managed either by a ReplicationController, or over a ReplicaSet / Deployment. In that case, annotate these resources to make the annotations reach the pods. In example of deployments, you must use the template section, like:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  # Unique key of the Deployment instance
  name: deployment-example
spec:
  # 3 Pods should exist at all times.
  replicas: 3
  # Keep record of 2 revisions for rollback
  revisionHistoryLimit: 2
  template:
    metadata:
      annotations:
        prometheus.io/scrape: 'true'
        prometheus.io/path: /websocket/metrics
        prometheus.io/port: '8080'
-- David Steiman
Source: StackOverflow