Why dockerd daemon is using too much memory??? I have several pods running that are using less than 4 GB of RAM and then I see that dockerd is using something like 6? Why this is happening? Is this acceptable? I mean, to manage pods my host should have all these resources? I have also included on my K8S cluster the logic to remove periodically old docker images not still used, so I suppose the problem is not related to old images.

Googling I can see that there are many people observing these kind of problem, related to possible leaks/issues on the version they are using. Anyhow I am observing this problem on two different machine with different versions:

My Virtual Machine:

Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:17:43 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:17:43 2017
 OS/Arch:      linux/amd64
 Experimental: false

My Azure K8S cluster:

arwadmin@k8s-master-B1FEF67B-0:~$ docker version

Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Tue Jan 10 20:38:45 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Tue Jan 10 20:38:45 2017
 OS/Arch:      linux/amd64

What it is very important for me is to understand the real amount of memory my cluster should be, due I am talking about a possible production environment where my pods shall be scaled in/out. Could you please help to understand this ?

I did another check now (docker stats), and in this moment on my cluster (with 3 agents), the agent0 is using more memory:

And the sum of all memory used by pods is something less than 4 GB. Why Dockerd daemon itself is using almost 7 GB on one agent and few MB on the other 2 agents ?