How to change CPU Limit for namespace kube-system
How to change CPU Limit for namespace kube-system in Azure Kubernetes? My pod could not be deployed successfully due to some pods from namespace kube-system using lots of resource.
Similar Questions
2 Answers
You can check for the resource quota for the namespace
kubectl describe quota
kubectl get quotaWhen several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources.
Resource quotas are a tool for administrators to address this concern.
A resource quota, defined by a ResourceQuota object, provides constraints that limit aggregate resource consumption per namespace. It can limit the quantity of objects that can be created in a namespace by type, as well as the total amount of compute resources that may be consumed by resources in that namespace.
apiVersion: v1
kind: List
items:
- apiVersion: v1
kind: ResourceQuota
metadata:
name: pods-high
spec:
hard:
cpu: "1000"
memory: 200Gi
pods: "10"
scopeSelector:
matchExpressions:
- operator : In
scopeName: PriorityClass
values: ["high"]https://kubernetes.io/docs/concepts/policy/resource-quotas/
i would also suggest checking the LimitRange
if LimitRange is supported into the cluster it might be assigning default resources to POD however editing Resourcequote will resolve your issue.
https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/
Posting this as community wiki out of comment, feel free to edit and expand
In short words, this is not possible to change limits for coreDNS and other critical resources located within kube-system namespace. (Technically it's possible to set custom values, but they will be overwritten shortly and initial state will get back to pre-defined one, below answer from microsoft how exactly it works).
There's a very similar question to it on microsoft question platform and this is the answer:
The deployment coredns runs system critical workload using the CoreDNS project for cluster DNS management and resolution with all 1.12.x and higher clusters. Reference.
If you do a kubectl describe deployment -n kube-system coredns, you will find a very interesting label addonmanager.kubernetes.io/mode=Reconcile
Now, addons with label addonmanager.kubernetes.io/mode=Reconcile will be periodically reconciled. Direct manipulation to these addons through apiserver is discouraged because addon-manager will bring them back to the original state. In particular:
Addon will be re-created if it is deleted.
Addon will be reconfigured to the state given by the supplied fields in the template file periodically.
Addon will be deleted when its manifest file is deleted from the $ADDON_PATH.
The $ADDON_PATH by default is set to /etc/kubernetes/addons/ on the control plane node(s).
For more information please check this document.
Since AKS is a managed Kubernetes Service you will not be able to access $ADDON_PATH. We strongly recommend against forcing changes to kube-system resources as these are critical for the proper functioning of the cluster.
Which was also confirmed in comment by OP:
just contacted MS support that we cannot change the limits form kube-system namespace.