We are using AKS version 1.19.11 and we are facing intermittent issues with the hpa, which will create the pods unnecessarly eventhough the pods cpu usage is very less.
Our application pods are light weighted and now we have added an hpa threshold of cpu -75% and (replica:- min6, max12) the pods were just using 9% of cpu. But sometimes the pods are getting autoscaled from replicas 5 to replicas suddenly as of a sudden and in another 2, 3 minutes will get back to 5 repicas itself.
Also during the rolling updates, its creating unexpected more pod.
So I would like to know whether my hpa settings have any issue on this for the abnormal behaviour and how can i fine-tune the thresholds so that
Make sure all your deployments have resources requests configured for sure.
HPA may misbehave if any of the deployment does not have resources requests configured.