Deploy HPA (Horizontal Pod Autoscaler) based custom metric (http request)?

4/4/2019

I am deploying my application using HPA algorithm based on http request. I follow this link. In the "Auto Scaling Based on Custom Metrics" part. I built successfully with their application. But when I deploy with my application, I get the error:

$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/http_requests" | jq .
Error from server (NotFound): the server could not find the metric http_requests for pods

In this part, they said that "The podinfo app exposes a custom metric named http_requests_total". So, how can my application expose a custom metric like that? Thank you so much!

-- nguyennd
kubernetes
prometheus

1 Answer

4/26/2019

You can find more information about this deployment here.
Another example how to build application for custom metrics you can find here and here.
In both cases the are using "Golang Client API"

In the example from your tutorial there is working application on port 9898
tcp 0 0 :::9898 :::* LISTEN 1/podinfo

If you are not sure if your application is working properly than please verify:

    kubectl get deploy,pods
    -- verify if your new deployment is working properly 
    kubectl describe <your pod>
    -- in case of any issues with your application:
    kubectl logs <your pod>
    -- if the port and an endpoint are the same as in the example
    curl <your pod ip (endpoint)>:9898/metrics 
       you should notice http_requests_total metrics: htp_requests_total{status="200"} 1926 
    -- for generall issues
    kubectl get events

Please share with your findings and results.

-- Hanx
Source: StackOverflow