I want to do Horizontal pod autoscaling on the basis of input request. I made query for prometheus which returns aggregated input tps.
sum(sum by (path) (rate(dapr_http_server_request_count{app_id="governor",path=~"/v1.0/invoke/app/method/interceptor/.*"}[10s])))
I want to use this output in kubernetes HPA.
I am using prometheus-adapter for the same. Prometheus configuration is as follows:
default: true
custom:
- seriesQuery: '{__name__=~"dapr_http_server_request_avg_.*",namespace!="",pod!=""}'
resources:
overrides:
namespace:
resource: namespace
pod:
resource: pod
name:
matches: "^(.*)_total"
as: "${1}_per_second"
metricsQuery: 'sum(sum by (path) (rate(dapr_http_server_request_count{app_id="governor",path=~"/v1.0/invoke/app/method/interceptor/.*"}[10s])))'
When i try to get the output in custom api of kubernetes it returns.
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/app/pods/*/dapr_http_server_request_avg_total" | jq .
Error from server (NotFound): the server could not find the metric dapr_http_server_request_avg_total for pods
You should check your prometheus-adapter config file, which makes me confused.
seriesQuery: dapr_http_server_request_avg_.*
metricsQuery: dapr_http_server_request_count
It is possible to get metrics dapr_http_server_request_count
from above series?
name:
matches: "^(.*)_total"
as: "${1}_per_second"
and here take care, your metric name seems to be renamed, you should find the right metric name for you query.
try this: kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1
you will see what your K8s Api-server actually get from Prometheus Adapter.