what's meaning the container_cpu_cfs_throttled_seconds_total metrics

11/13/2018

cadvisor has two metrics container_cpu_cfs_throttled_seconds_total and container_cpu_cfs_throttled_periods_total

I have confuse what does that means ..

I have found about two explain:

  1. container run with cpu limit, when container cpu over limit , the container will be "throttled" and add time to container_cpu_cfs_throttled_seconds_total

    that means :
     (1). only container cpu over limit, rate(container_cpu_cfs_throttled_seconds_total) > 0. 
     (2). we can use this metrics to alert container cpu over limit ... 
  2. when host in heavy cpu pressure, it will "throttled" container with POD QoS(Guaranteed > Burstable > Best-Effort) ...

    that means :
     (1). container_cpu_cfs_throttled_seconds_total will add has no relate with how many cpu container used and cpu limit ..
     (2). this metrics can not to alert container cpu over limit .. 
-- gpl
kubernetes-pod

1 Answer

1/18/2020

Lets say httpbin container running on machine1. Lets say httbin has a limit set in it's deployment to use maximum of 1 CPU. And machine1 has 2 CPUs. It makes httpbin to use half the available.

If httpbin container is trying to use more than 1 CPU, kubernetes will not kill the container. It will throttle it. If it is happening frequently, you may want to get alerted on that and fix the deployment. Another scenario is, if there are multiple containers in machine1 and if there is a lack of CPU resource, then it will throttle all containers it has.

container_cpu_cfs_throttled_seconds_total is the Total time duration the container has been throttled in seconds. container_cpu_cfs_throttled_periods_total is the Number of throttled period intervals

-- ffran09
Source: StackOverflow