Could someone please help me understand the relationship between HorizontalPodAutoscaler and the number of nodes / pods.
If the autoscaler adds more pods than a node can handle, will the new pods just go into a 'pending' state or will it automatically trigger a new node / vCPU on say... Google Kubernetes Engine to spin up?
Cheers, Clement
An HPA will essentially create a minimum number of pods and will increase to the maximum number of pods according to a defined metric like CPU percentage on the pod. This is just at the Kubernetes level, it won't scale your nodes.
To scale your nodes you need to use a different autoscaler which could be defined according to metrics on your servers. Examples of these are the Amazons Autoscaling Groups (ASGs). There are equivalents in GCP (server pools) and Azure.
So if you don't have your server autoscaler set up correctly or you don't have enough capacity in your cluster you could run into cases where Kubernetes HPA(s) want to scale and don't find enough server capacity. If that happens then you'll have a lot of pods go into "Pending" state.
Keep in mind that Kubernetes also has Vertical Pod Autoscaler and the takes care of autoscaling the resources inside the pods, such as memory and cpu percentage.
This link has good information about all the autoscaling methods.
Hope it helps.