Why does Kubernetes v1.12 no longer need an upscale delay?

8/28/2019

In the Kubernetes documentation for horizontal pod autoscalers it states as of version 1.12 a "new algorithmic update removes the need for the upscale delay"

I have searched for information on this change including going through the v1.12 change log. The change I see mentioned is the polling frequency from 30 seconds to 15 seconds.

There are also some discussions about adding HPA configurations for scale delay.

What was the change that removed the need for upscale delay?

-- leemicw
autoscaling
kubernetes

1 Answer

8/28/2019

There are several changes (quoted from the release notes):

  • Replace scale up forbidden window with disregarding CPU samples collected when pod was initializing. (#67252, @jbartosik)
  • Speed up HPA reaction to metric changes by removing scale up forbidden window. (#66615, @jbartosik)

    • Scale up forbidden window was protecting HPA against making decision to scale up based on metrics gathered during pod initialisation (which may be invalid, for example pod may be using a lot of CPU despite not doing any "actual" work).
    • To avoid that negative effect only use per pod metrics from pods that are:
    • ready (so metrics about them should be valid), or
    • unready but creation and last readiness change timestamps are apart more than 10s (pods that have formerly been ready and so metrics are in at least some cases (pod becoming unready because of overload) very useful).
  • Horizontal Pod Autoscaler default update interval has been increased from 30s to 15s, improving HPA reaction time for metric changes. (#68021, @krzysztof-jastrzebski)

  • Stop counting soft-deleted pods for scaling purposes in HPA controller to avoid soft-deleted pods incorrectly affecting scale up replica count calculation. (#67067, @moonek)

  • To avoid soft-deleted pods incorrectly affecting scale up replica count calculations, the HPA controller will stop counting soft-deleted pods for scaling purposes. (#67067, @moonek) (same as above)

This is a related change (quoted from the release notes):

  • Replace scale down forbidden window with scale down stabilization window. Rather than waiting a fixed period of time between scale downs HPA now scales down to the highest recommendation it during the scale down stabilization window. (#68122, @krzysztof-jastrzebski)

More documentation related to that change is here.

-- Rico
Source: StackOverflow