Cannot curl kubelet read-only port

10/26/2017

I have a heapster pod running on one of the nodes in my Kubernetes cluster. It is able to get http://<node-with-heapster-pod>:10255/stats/summary just fine, but whenever it runs the same get request on another node, it cannot. When I run curl from within any given node I can access that port, but when I curl any node from another machine I get the following error:

Failed to connect to 128.180.120.229 port 10255: No route to host

The following is the netstat output for all ports on which the kubelet is listening:

netstat -ap | grep -i "listen" | grep "kubelet"
tcp        0      0 localhost:10248         0.0.0.0:*               LISTEN 7562/kubelet        
tcp6       0      0 [::]:4194               [::]:*                  LISTEN     7562/kubelet        
tcp6       0      0 [::]:10250              [::]:*                  LISTEN 7562/kubelet
tcp6       0      0 [::]:10255              [::]:*                  LISTEN      7562/kubelet      
unix  2      [ ACC ]     STREAM     LISTENING     621349   7562/kubelet       /var/run/dockershim.sock

I apologize for the messy last column. Any ideas why this may be? My iptables rules are set up to accept all incoming connections, and any node can ping port 10250 fine, just not 10255.

-- Ryan
kubelet
kubernetes

1 Answer

10/27/2017

you may not have ip_forward enabled on your system. can you check this settings?

 sysctl -n net.ipv4.ip_forward
-- sfgroups
Source: StackOverflow