K
Q

Kubernetes: kubectl top nodes/pods not working

10/8/2018

When I do kubectl top pods/nodes I am getting the following error:

metrics not available yet

When I check the logs I am getting

$ kubectl logs metrics-server-7df4c4484d-h75wr -n kube-system -c metrics-server

I1008 12:02:55.159069       1 serving.go:273] Generated self-signed cert(apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
[restful] 2018/10/08 12:02:56 log.go:33: [restful/swagger] listing is available at https://:443/swaggerapi
[restful] 2018/10/08 12:02:56 log.go:33: [restful/swagger] https://:443/swaggerui/ is mapped to folder /swagger-ui/
I1008 12:02:56.358063       1 serve.go:96] Serving securely on [::]:443
E1008 12:03:04.225519       1 reststorage.go:101] unable to fetch node metrics for node "hpa-test": no metrics known for node "hpa-test"
E1008 12:03:07.619489       1 reststorage.go:101] unable to fetch node metrics for node "hpa-test": no metrics known for node "hpa-test"

Also, I am able to ping to hpa-test node from:

$ kubectl exec -it  metrics-server-7df4c4484d-h75wr -n kube-system sh

Also, I have tried looking for solution everywhere but nothing fixed the issue

-- Mallikarjun Br
docker
kubelet
kubernetes
kubernetes-metrics

Similar Questions

Kubernetes: exec via WebSocket API; close stdin?
How to connect a database to a backend in minicube?
kubernetes hpa issue with metrics-server
K8s: what is the difference between Security Context and Security Policy?
Doing kubectl rollout undo causing current and previous replicasets persists

4 Answers

10/21/2019

These issue arises because you don't have metric server enabled in your cluster

The top command will only be running if the metric server installed on your cluser

Download metric server in Kubernetes API from the git

Step:1 git clone https://github.com/kubernetes-incubator/metrics-server.git

Step:2 cd metrics-server/

Step:3 Deploy

Kubernetes 1.7 kubectl create -f deploy/1.7/

Kubernetes 1.8

kubectl create -f deploy/1.8+/

Setp :4 Check status of metric server pod

 kubectl get po -n kube-system |grep metrics

Step5: Now your top command will be running

kubectl top pod --all-namespaces
-- UDIT JOSHI
Source: StackOverflow
10/23/2019

Edit the deployment of metrics-server and add the following to arguments to its container.

--kubelet-insecure-tls
--kubelet-preferred-address-types=InternalIP

For example,

kind: Deployment
metadata:
  name: metrics-server
  ...
spec:
  template:
    spec:
      ...
      containers:
      - name: metrics-server
        image: k8s.gcr.io/metrics-server-amd64:v0.3.6
        args:
          - --kubelet-insecure-tls
          - --kubelet-preferred-address-types=InternalIP
-- wizawu
Source: StackOverflow
12/13/2018

There are two ways to fix this problem:

1) using heapster : installing heapster will allow 'kubectl top nodes' to work out of the box. However heapster has been deprecated, so you probably should switch to metrics-server.

2) using metrics-server : unfortunately it may not work out of the box when installing metrics-server... in case it doesn't, you need to update the end of the metrics-server-deployment.yaml (1.8+) file you used for installation and add a command section with the right parameters as follow:

containers:
- name: metrics-server
  image: k8s.gcr.io/metrics-server-amd64:v0.3.1
  imagePullPolicy: Always
  volumeMounts:
  - name: tmp-dir
    mountPath: /tmp

  command:
      - /metrics-server
      - --kubelet-insecure-tls
      - --kubelet-preferred-address-types=InternalIP

then simply apply the changes:

kubectl apply -f metrics-server-deployment.yaml

you should then be able to get results with

kubectl top nodes

and

kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes"

-- aregnier
Source: StackOverflow
10/8/2018

Need add flags for metrics-sever:

--kubelet-insecure-tls=true
--kubelet-port={YOU_KUBELET_PORT}
--kubelet-preferred-address-types=InternalIP
--v=5
--logtostderr
-- Arslanbekov
Source: StackOverflow