HPA not running in custom namespaces

12/6/2019

I am trying to create an HPA resource in my Kubernetes environment.

To give you context, this is a GKE cluster running in Google GCP cloud. There's no custom or tweaked setting.

Client: GitVersion:"v1.13.9-2+4a03651a7e7e04", GoVersion:"go1.11.5", 
Server: GitVersion:"v1.13.11-gke.14", GoVersion:"go1.12.11b4",

When I deploy my HPA in the "default" namespace everything is running fine, it can find its attached deployment and fetch the CPU metrics.

However, when I deploy exactly the same resources to a specific namespace, it is unable to find the CPU metrics.

As an exemple :

kubectl get hpa -n automl
NAME                                                                REFERENCE                                  TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/automl-icn8751773907515604992   Deployment/automl-icn8751773907515604992   <unknown>/80%   1         5         1          139m

Targets field stay in <unknown>/80% even after several minutes, meaning something is wrong

When I want to know more about the HPA :

kubectl describe hpa automl-icn8751773907515604992 -n automl

I get this :

Name:                     automl-icn8751773907515604992
Namespace:                automl
Labels:                   <none>
Annotations:              autoscaling.alpha.kubernetes.io/conditions:
                            [{"type":"AbleToScale","status":"True","lastTransitionTime":"2019-12-06T10:29:22Z","reason":"SucceededGetScale","message":"the HPA control...
CreationTimestamp:        Fri, 06 Dec 2019 11:29:12 +0100
Reference:                Deployment/automl-icn8751773907515604992
Target CPU utilization:   80%
Current CPU utilization:  <unknown>%
Min replicas:             1
Max replicas:             5
Deployment pods:          1 current / 0 desired
Events:
  Type     Reason                   Age                   From                       Message
  ----     ------                   ----                  ----                       -------
  Warning  FailedGetResourceMetric  23s (x553 over 140m)  horizontal-pod-autoscaler  missing request for cpu

I can detect something failed because of the last line, but since this is the only error message I get, I am unable to dig more.

I already tried to parse documentation, and issues forums, but no one could answer to my question. I suspect this could be an RBAC issue but I have no clue.

Have you ever met this issue and how did you resolve it ?

Thank you in advance.

-- Vincent VAN HOLLEBEKE
kubernetes

0 Answers