How to decrease CPU load aggregation to 1s instead of 10s in cAdvisor?

8/2/2018

I'm using Kubernetes latest version (bare-metal on a multi-node cluster), and I can get cAdivsor metrics using Kubernetes metrics API. I can see cAdvisor provides container_cpu_load_average_10s metric which from its name, it is the aggregation of CPU loads in 10s.

Is there any way I can reduce this aggregation/granularity time to 1s? (having something like container_cpu_load_average_1s)

P.S: API Endpoint that I'm using is https://<my_k8s_ip>:6443/api/v1/nodes/kubernetes3/proxy/metrics/cadvisor

UPDATE: I just find out there is a --housekeeping_interval parameter but need to find out if it is the way to go, and if so then how can I change it in my current Kubernetes deployment. Any suggestion would be appreciated.

-- Michel Gokan
cadvisor
docker
kubernetes
monitoring
performance

2 Answers

8/16/2018

We should set two arguments in 10-kubeadm.conf:

  1. --enable-load-reader
  2. --housekeeping_interval=1s
-- Michel Gokan
Source: StackOverflow

8/3/2018

Actually you don't need to update your deployment. --housekeeping_interval is parameter of kubelet . You need just add --housekeeping_interval=1s to startup script of kubelet. Config file for start kubelet is usually located in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf. Just update the string:

ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS --housekeeping_interval=1s

Add the --housekeeping_interval=1s at the end, and restart kubelet. You need to do this on all nodes.

-- Nick Rak
Source: StackOverflow