I've used Kubeadm to bootstrap a cluster, and added weave-net with
kubectl apply -f https://git.io/weave-kube
and I have everything running, but I can't "see" any of the assigned IP's within the cluster.
So:
[centos@atomic01 ~]$ kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE default hello-2533203682-b5usp 1/1 Running 0 13m 10.42.0.0 atomic03 default test-701078429-ely8s 1/1 Running 1 3h 10.40.0.1 atomic02 kube-system dummy-2088944543-6i81l 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system etcd-atomic01 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system kube-apiserver-atomic01 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system kube-controller-manager-atomic01 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system kube-discovery-982812725-c1kkw 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system kube-dns-2247936740-nrszw 3/3 Running 2 5h 10.32.0.2 atomic01 kube-system kube-proxy-amd64-0y8ik 1/1 Running 1 5h 192.168.150.152 atomic03 kube-system kube-proxy-amd64-57y4o 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system kube-proxy-amd64-mjpik 1/1 Running 1 5h 192.168.150.151 atomic02 kube-system kube-proxy-amd64-sh3ej 1/1 Running 1 5h 192.168.150.153 atomic04 kube-system kube-scheduler-atomic01 1/1 Running 0 5h 192.168.150.150 atomic01 kube-system kubernetes-dashboard-3095304083-xwuw8 1/1 Running 1 2h 10.38.0.0 atomic04 kube-system weave-net-edur9 2/2 Running 0 1m 192.168.150.151 atomic02 kube-system weave-net-l9xp3 2/2 Running 0 1m 192.168.150.150 atomic01 kube-system weave-net-sjpui 2/2 Running 0 1m 192.168.150.153 atomic04 kube-system weave-net-xu7j5 2/2 Running 0 1m 192.168.150.152 atomic03
I should be able to ping the other nodes, but
[centos@atomic01 ~]$ kubectl exec test-701078429-ely8s -- ping 10.42.0.0 PING 10.42.0.0 (10.42.0.0) 56(84) bytes of data. From 10.40.0.1 icmp_seq=1 Destination Host Unreachable From 10.40.0.1 icmp_seq=2 Destination Host Unreachable From 10.40.0.1 icmp_seq=3 Destination Host Unreachable
Of course, this works:
[centos@atomic01 ~]$ kubectl exec test-701078429-ely8s -- ping 192.168.150.150 PING 192.168.150.150 (192.168.150.150) 56(84) bytes of data. 64 bytes from 192.168.150.150: icmp_seq=1 ttl=63 time=0.484 ms 64 bytes from 192.168.150.150: icmp_seq=2 ttl=63 time=0.448 ms
I've run out of ideas, any clues on things to test or look out for would be much appreciated. [Running on Centos 7 Atomic VM's]
Hmm, does any of the test or hello Pods expose any ports? Can you access a port on the hello pod from the test pod?
Normally, I don't think ping should work (unless you have a Pod that processes icmp requests), so I actually don't think there's anything wrong here.
Just run a Pod (preferably from a Deployment) that exposes a port maybe 80 for simplicity, set containerPort: 80 and you should be able to curl that Pod IP successfully. Also consider making a Service with a stable IP that loadbalances requests to matching Pods, as Pods might come and go.
Hope it helps!