K
Q

Question

How to expose kubernetes metric server api to curl from inside the pod?

11/18/2019

I am using metric server to get the usage of my Kubernetes cluster. But in order to use it from outside the host, I need to use "kubectl proxy". But i don't want to do that as it is not intended to run on background. I want it to be run as a service continuously

How can i achieve these

expected output curl clusterip:8001/apis/metrics.k8s.io/v1beta1/nodes

{
  "kind": "NodeMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
  },
  "items": [
    {
      "metadata": {
        "name": "manhattan-master",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/manhattan-master",
        "creationTimestamp": "2019-11-15T04:26:47Z"
      },
      "timestamp": "2019-11-15T04:26:33Z",
      "window": "30s",
      "usage": {
        "cpu": "222998424n",
        "memory": "3580660Ki"
      }
    }
  ]

I tried by using LoadBalancig service metrics-server-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: metrics-server
  namespace: kube-system
  labels:
    kubernetes.io/name: "Metrics-server"
    kubernetes.io/cluster-service: "true"
spec:
  selector:
    k8s-app: metrics-server
  ports:
  - port: 443
    protocol: TCP
    targetPort: main-port
  externalTrafficPolicy: Local
  type: LoadBalancer

kubectl describe service metrics-master -n kube-system

[root@manhattan-master 1.8+]# kubectl describe service metrics-server -n kube-system
Name:                     metrics-server
Namespace:                kube-system
Labels:                   kubernetes.io/cluster-service=true
                          kubernetes.io/name=Metrics-server
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"kubernetes.io/cluster-service":"true","kubernetes.io/name":"Me...
Selector:                 k8s-app=metrics-server
Type:                     LoadBalancer
IP:                       10.110.223.216
Port:                     <unset>  443/TCP
TargetPort:               main-port/TCP
NodePort:                 <unset>  31043/TCP
Endpoints:                10.32.0.7:4443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     32208
Events:                   <none>

-- UDIT JOSHI

kubectl

kubernetes

kubernetes-service

1 Answer

11/19/2019

This is possible by creating a new service to expose the Metrics Server. Your Metrics Server Service should look like this:

apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/name: Metrics-server-ext
  name: metrics-server-ext
  namespace: kube-system
  selfLink: /api/v1/namespaces/kube-system/services/metrics-server
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
  sessionAffinity: None
  type: LoadBalancer

If you try to access this service you will face some problems with authorization and you need to do some things to give all necessary authorizations.

After creating the service you will need to create a Cluster Role Binding so our service can have access to the data:

$ kubectl create clusterrolebinding node-admin-default-svc --clusterrole=cluster-admin --serviceaccount=default:default

Before running curl command we need to get the token so we can pass this on our curl command:

$ TOKEN=$(kubectl get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='default')].data.token}"|base64 --decode)

Get your service external IP:

kubectl get svc/metrics-server-ext -n kube-system -o jsonpath='{..ip}'

Your curl command should pass the Token key to get Authorization:

curl -k https://34.89.228.98/apis/metrics.k8s.io/v1beta1/nodes --header "Authorization: Bearer $TOKEN" --insecure

Sample output:

{
 "kind": "NodeMetricsList",
 "apiVersion": "metrics.k8s.io/v1beta1",
 "metadata": {
   "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
 },
 "items": [
   {
     "metadata": {
       "name": "gke-lab-default-pool-993de7d7-ntmc",
       "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/gke-lab-default-pool-993de7d7-ntmc",
       "creationTimestamp": "2019-11-19T10:26:52Z"
     },
     "timestamp": "2019-11-19T10:26:17Z",
     "window": "30s",
     "usage": {
       "cpu": "52046272n",
       "memory": "686768Ki"
     }
   },
   {
     "metadata": {
       "name": "gke-lab-default-pool-993de7d7-tkj9",
       "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/gke-lab-default-pool-993de7d7-tkj9",
       "creationTimestamp": "2019-11-19T10:26:52Z"
     },
     "timestamp": "2019-11-19T10:26:21Z",
     "window": "30s",
     "usage": {
       "cpu": "52320505n",
       "memory": "687252Ki"
     }
   },
   {
     "metadata": {
       "name": "gke-lab-default-pool-993de7d7-v7m3",
       "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/gke-lab-default-pool-993de7d7-v7m3",
       "creationTimestamp": "2019-11-19T10:26:52Z"
     },
     "timestamp": "2019-11-19T10:26:17Z",
     "window": "30s",
     "usage": {
       "cpu": "45602403n",
       "memory": "609968Ki"
     }
   }
 ]
}

EDIT:

You can also optionally access it from your pods since you created a Cluster Role Binding in your default Service Account with cluster-admin role.

As example, create a pod from a image that includes curl command:

$ kubectl run bb-$RANDOM --rm -i --image=ellerbrock/alpine-bash-curl-ssl --restart=Never --tty -- /bin/bash

Than you need to exec into you pod and run:

$ curl -k -X GET https://kubernetes.default/apis/metrics.k8s.io/v1beta1/nodes --header "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" --insecure

Here we are passing the same TOKEN mentioned before in a complete different way.

-- mWatney

Source: StackOverflow

KQ

How to expose kubernetes metric server api to curl from inside the pod?

Similar Questions

1 Answer

K
Q