Playing with cluster autoscaler I've noticed that scale-down is not working due to standard k8s pods:
Fast evaluation: node aks-nodepool1-37748512-0 cannot be removed: non-daemonset, non-mirrored, non-pdb-assignedkube-system pod present: kube-dns-v20-8748686c5-27psnWhat is a proper PodDisruptionBudget for kube-dns and are there any best practices for standard system POD PDBs? Why aren't they configured by default?
Inside Kubernetes docs about Disruptions we can read:
An Application Owner can create a
PodDisruptionBudgetobject (PDB) for each application. A PDB limits the number pods of a replicated application that are down simultaneously from voluntary disruptions. For example, a quorum-based application would like to ensure that the number of replicas running is never brought below the number needed for a quorum. A web front end might want to ensure that the number of replicas serving load never falls below a certain percentage of the total.
You can see examples on how to correctly enable, tune and disable PodDisruptionBudget for kube-dns inside Kubernetes docs for Autoscale the DNS Service in a Cluster.
Also Marton Sereg wrote a good article about Draining Kubernetes nodes, in which he explains how does draining works and what's happening inside the cluster.
As for configuration defaults I was able to find this discussion Reasonable defaults with eviction and PodDisruptionBudget #35318.