What is the interval of kube-controller-manager control loop?

2/12/2022

I see this in Kubernetes doc,

In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state.

Also this,

The Deployment controller and Job controller are examples of controllers that come as part of Kubernetes itself ("built-in" controllers).

But, I couldn't find how does the control loop work. Does it check the current state of the cluster every few seconds? If yes, what is the default value?

I also found something interesting here,

https://stackoverflow.com/questions/55453072/what-is-the-deployment-controller-sync-period-for-kube-controller-manager

-- karthikeayan
kube-controller-manager
kubelet
kubernetes
loops
reconcile

1 Answer

2/14/2022

I would like to start explaining that the kube-controller-manager is a collection of individual control processes tied together to reduce complexity.

Being said that, the control process responsible for monitoring the node's health and a few other parameters is the Node Controller, and it does that by reading the Heartbeats sent by the Kubelet agent in the nodes.

According to the Kubernete's documentation:

For nodes there are two forms of heartbeats:

  • updates to the .status of a Node
  • Lease objects within the kube-node-lease namespace. Each Node has an associated Lease object.

Compared to updates to .status of a Node, a Lease is a lightweight resource. Using Leases for heartbeats reduces the performance impact of these updates for large clusters.

The kubelet is responsible for creating and updating the .status of Nodes, and for updating their related Leases.

  • The kubelet updates the node's .status either when there is change in status or if there has been no update for a configured interval. The default interval for .status updates to Nodes is 5 minutes, which is much longer than the 40 second default timeout for unreachable nodes.
  • The kubelet creates and then updates its Lease object every 10 seconds (the default update interval). Lease updates occur independently from updates to the Node's .status. If the Lease update fails, the kubelet retries, using exponential backoff that starts at 200 milliseconds and capped at 7 seconds.

As for the Kubernetes Objects running in the nodes:

Kubernetes objects are persistent entities in the Kubernetes system. Kubernetes uses these entities to represent the state of your cluster. Specifically, they can describe:

  • What containerized applications are running (and on which nodes)
  • The resources available to those applications
  • The policies around how those applications behave, such as restart policies, upgrades, and fault-tolerance

A Kubernetes object is a "record of intent"--once you create the object, the Kubernetes system will constantly work to ensure that object exists. By creating an object, you're effectively telling the Kubernetes system what you want your cluster's workload to look like; this is your cluster's desired state.

Depending on the Kubernetes Object, the controller mechanism is responsible for maintaining its desired state. The Deployment Object for example, uses the Replica Set underneath to maintain the desired described state of the Pods; while the Statefulset Object uses its own Controller for the same purpose.

To see a complete list of Kubernetes Objects managed by your cluster, you can run the command: kubectl api-resources

-- Gabriel Robledo Ahumada
Source: StackOverflow