K
Q

HPA using Kafka Exporter in on premise Kubernetes cluster

8/5/2019

I had been trying to implement Kubernetes HPA using Metrics from Kafka-exporter. Hpa supports Prometheus, so we tried writing the metrics to prometheus instance. From there, we are unclear on the steps to do. Is there an article where it will explain in details ?

I followed https://medium.com/google-cloud/kubernetes-hpa-autoscaling-with-kafka-metrics-88a671497f07

for same in GCP and we used stack driver, and the implementation worked like a charm. But, we are struggling in on-premise setup, as stack driver needs to be replaced by Prometheus

-- Jagadheeswaran Mohan
apache-kafka
kafka-consumer-api
kubernetes
kubernetes-hpa

Similar Questions

Serve one path internally and a different one externally
How to handle secrets in ConfigMaps?
Reduce Cloud Run on GKE costs
Default CNI in Minikube

2 Answers

8/25/2019

When I implemented Kubernetes HPA using Metrics from Kafka-exporter I had a few setbacks which I solved doing the following:

  1. I deployed the kafka-exporter container as a sidecar to the pods I wanted to scale. I found that the HPA scales the pod it gets the metrics from.

  2. I used annotations to make Prometheus scrape the metrics from the pods with exporter.

  3. Then I verified that the kafka-exporter metrics are getting to Prometheus. If it's not there you can't advance further.

  4. I deployed prometheus adapter using its helm chart. The adapter will "translate" Prometheus's metrics into custom Metrics Api, which will make it visible to HPA.

  5. I made sure that the metrics are visible in k8s by executing kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 from one of the master nodes.
  6. I created an hpa with the matching metric name.

Here is a complete guide explaining how to implement Kubernetes HPA using Metrics from Kafka-exporter

Please comment if you have more questions

-- balaganAtomi
Source: StackOverflow
8/5/2019

In order to scale based on custom metrics, Kubernetes needs to query an API for metrics to check for those metrics. That API needs to implement the custom metrics interface.

So for Prometheus, you need to setup an API that exposes Prometheus metrics through the custom metrics API. Luckily, there already is an adapter.

-- Blokje5
Source: StackOverflow