Kubernetes - dockerd high CPU usage

7/9/2019

My server process running on k8s deployed by aws eks. Recently, I found CPU usage is high in one node, and found the dockerd consume high CPU.

top - 11:36:36 up 14 days,  8:44,  2 users,  load average: 8.46, 9.09, 9.35
Tasks: 201 total,   7 running, 129 sleeping,   0 stopped,   0 zombie
%Cpu(s): 71.4 us,  9.6 sy,  0.0 ni, 17.0 id,  0.0 wa,  0.0 hi,  1.9 si,  0.1 st
KiB Mem : 15392012 total,  2815656 free,  2888368 used,  9687988 buff/cache
KiB Swap:        0 total,        0 free,        0 used. 12844852 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
29907 root      20   0 1180260   1.1g  21832 R 327.9  7.4   5052:50 server
 5028 root      20   0 6406920   1.0g  39156 S 242.9  6.7   8380:11 dockerd
32492 root      20   0 2558816 200264  55240 S  40.9  1.3 300:46.66 agent

node info

 Kernel Version:             4.14.106-97.85.amzn2.x86_64
 OS Image:                   Amazon Linux 2
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://18.6.1
 Kubelet Version:            v1.12.7
 Kube-Proxy Version:         v1.12.7

And also I check memory usage and disk usage is well in this node.

Why CPU usage of dockerd is high? or how to debug it?

-- zangw
amazon-eks
docker
kubernetes

0 Answers