I have kubernetes 1.11 cluster with 26 nodes and 3 master all servers running on Centos 7 OS, its running for a while. recently I noticed kube-api server establishing too many connection to node kubelet port 10250
. today its used all the ports on the primary master, helm command started failing. I have restart some nodes to clean up the ports
tcp 0 0 xx.xx.xx3.53:50863 xx.xx.xx4.94:10250 ESTABLISHED
tcp 0 0 xx.xx.xx3.53:60531 xx.xx.xx3.62:10250 ESTABLISHED
tcp 0 0 xx.xx.xx3.53:42625 xx.xx.xx3.63:10250 ESTABLISHED
tcp 0 0 xx.xx.xx3.53:56703 xx.xx.xx3.61:10250 ESTABLISHED
tcp 0 0 xx.xx.xx3.53:49426 xx.xx.xx3.64:10250 ESTABLISHED
all the nodes it used 50 thousand ports
netstat -an |grep -ci estab
51163
I like to know why kube-api creating too many connection to kubelet. how to resolve this issue.
Thanks SR
As spotted by @Ijaz Ahmad Khan, it's a known issue affecting v1.11.1, reported already as bug on github under the bug report #67382. From what I can see it`s already fixed in upper Kubernetes releases.