Horizontal Pods Autoscaling (HPA) on GKE based on GPU utilization

2/20/2020

I am trying to achieve Horizontal Pods Autoscaling (HPA) in GCP based on GPU utilization. My hpa.yaml file looks like this:

kind: HorizontalPodAutoscaler
metadata:
  name: my-hpa
spec:
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: External
    external:
     metricName: kubernetes.io|container|accelerator|duty_cycle
      targetAverageValue: 10
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-container-worker

After I run kubectl create -f hpa.yaml , I see the following error in the GCP.

enter image description here

On the HPA, it says unable to read all metrics.

In stack driver monitoring I have created a dashboard with the metric mentioned above.

Has anyone come across this issue?

-- Suraj Acharya
google-compute-engine
google-kubernetes-engine
hpa
kubernetes
kubernetes-pod

1 Answer

2/25/2020

Summarizing our conversation in comments:

To make use of GCP metrics in kubernetes, Custom Metrics Stackdriver Adapter had to be deployed. This is why HPA was unable to read the metrics and therefore was throwing errors.

-- HelloWorld
Source: StackOverflow