K
Q

(Kubernetes EKS) - metrics-server not available

11/22/2019

I'm trying to enable autoscaling policy for the pods in my 1.13 EKS cluster. I have been bumping into the same problem which is that the metrics-server is never available.

kubectl get deployment metrics-server -n kube-system
NAME             DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
metrics-server   1         1         1            0           9s

I have cloned the repo necessary on my own machine then kubectl apply -f deploy/1.8+. I have added those lines to the deployment as well:

       args:
          - --cert-dir=/tmp
          - --secure-port=4443
          - --kubelet-insecure-tls
          - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        command:
          - /metrics-server
          - --metric-resolution=30s
          - --requestheader-allowed-names=aggregator
          - --kubelet-insecure-tls
          - --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP

I have tried the args, then the command part both separately, then together. I have as well taken off heapster from my cluster (kubectl delete -f this and that).

Thank you for your help

Edit: When I run kubectl describe deployment metrics-server -n kube-system, here is what I see:

Name:                   metrics-server
Namespace:              kube-system
CreationTimestamp:      Fri, 22 Nov 2019 16:08:14 +0000
Labels:                 k8s-app=metrics-server
Annotations:            deployment.kubernetes.io/revision: 1
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"k8s-app":"metrics-server"},"name":"metrics-server","na...
Selector:               k8s-app=metrics-server
Replicas:               1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:           k8s-app=metrics-server
  Service Account:  metrics-server
  Containers:
   metrics-server:
    Image:        k8s.gcr.io/metrics-server-amd64:v0.3.6
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:
      /tmp from tmp-dir (rw)
  Volumes:
   tmp-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      False   MinimumReplicasUnavailable
  Progressing    False   ProgressDeadlineExceeded
OldReplicaSets:  <none>
NewReplicaSet:   metrics-server-6fbb7b8994 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  10m   deployment-controller  Scaled up replica set metrics-server-6fbb7b8994 to 1
-- shrimpy
amazon-eks
aws-eks
kubernetes

Similar Questions

Kubernetes - services without selector
some backup storage locations are invalid: backup store for location "default" is invalid:
Airflow scheduler with kubernetes executor fails :- Unknown error in KubernetesJobWatcher
helm override list values with --set in Azure DevOps
RabbitMQ cannot start after upgrading Azure Kubernetes Service (AKS)
Kubernetes - service type LoadBalancer to use specific ip address every time deployed in AKS

1 Answer

11/27/2019

Found out what was wrong, if you end up with the same problem than me, here is the procedure: Deploy metrics-server with those in the deployment:

- name: metrics-server
        image: k8s.gcr.io/metrics-server-amd64:v0.3.6
        args:
          - --cert-dir=/tmp
          - --secure-port=4443
          - --kubelet-insecure-tls

Apply this hpa

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
 name: cpu-trial
 namespace: trial
spec:
 scaleTargetRef:
   apiVersion: apps/v1beta1
   kind: Deployment
   name: cpu-trial
 minReplicas: 3
 maxReplicas: 5
 metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 85
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 85

Apply these lines in the deployment

        image: #{image}
        resources:
          limits:
            cpu: 200m
            memory: "65Mi"
          requests:
            cpu: 100m
            memory: "40Mi"

That should be it!

-- shrimpy
Source: StackOverflow