I've been working at this, and I'm not making any progress.
The issue is that when I create a service out of a deployment, the ClusterIp that's created for the service isn't accessible within MiniKube as I expect it should be.
I can verify that it's not accessible by sshing into a different pod than the one I've exposed, and pinging the IP of the service.
kubectl expose deployment/foo --target-port=2500
This creates the service at 10.0.0.5, which routes to ${foo's IP}:2500
kubectl exec -it bar-5435435-sadasf -- bash root@bar-5435435-sadasf:/# ping 10.0.0.5
PING 10.0.0.5 (10.0.0.5): 56 data bytes ^C--- 10.0.0.5 ping statistics --- 8 packets transmitted, 0 packets received, 100% packet loss
I have no issue pinging the pod IP ($foo's IP), but that's not what I want to do.
I've done enough reading to know that the issue is likely related to proxy.go which seems to be the kube-proxy equivalent in Minikube.
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies https://github.com/kubernetes/minikube/blob/master/pkg/localkube/proxy.go
I've checked out the Minikube logs and grepped for anything containing "proxy", and it seems this might point to the issue, but I don't know how to solve it.
My latest proxy logs:
Sep 06 18:13:06 minikube localkube[3373]: Starting proxy... Sep 06 18:13:06 minikube localkube[3373]: Waiting for proxy to be healthy... Sep 06 18:13:07 minikube localkube[3373]: proxy is ready! Sep 06 18:46:47 minikube localkube[3373]: E0906 18:46:47.742324 3373 proxy.go:207] Error proxying data from backend to client: write tcp 192.168.99.101:8443->192.168.99.1:58760: write: broken pipe Sep 06 19:11:41 minikube localkube[3373]: E0906 19:11:41.077014 3373 proxy.go:193] Error proxying data from client to backend: write tcp 127.0.0.1:44180->127.0.1.1:10250: write: broken pipe Sep 06 19:11:41 minikube localkube[3373]: E0906 19:11:41.077220 3373 proxy.go:207] Error proxying data from backend to client: write tcp 192.168.99.101:8443->192.168.99.1:45586: write: broken pipe Sep 06 19:22:07 minikube localkube[3373]: E0906 19:22:07.214287 3373 proxy.go:207] Error proxying data from backend to client: write tcp 192.168.99.101:8443->192.168.99.1:51558: write: broken pipe Sep 06 19:22:07 minikube localkube[3373]: E0906 19:22:07.214459 3373 proxy.go:193] Error proxying data from client to backend: write tcp 127.0.0.1:45184->127.0.1.1:10250: write: broken pipe
Does anyone have an idea what the issue is and how to fix it?
A Service gets its Virtual IP address using the ClusterIP. That IP address is used for communicating with the Service and is accessible only within the cluster.
Make sure that you connect to a service not only by its IP but also with the port that service exposed.
In your case:
$ kubectl expose deployment/foo --port=3030 --target-port=2500
$ kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
foo 10.0.0.73 <nodes> 3030/TCP 1mNow the service is reacheable from within a cluster:
$ minikube ssh
$ curl 10.0.0.73:3030
Hello World!
$ exit
$ kubectl exec -i -t bar-j26rd /bin/sh
$ curl 10.0.0.73:3030
Hello World!
$ exit