I'm playing a bit with the Google Cloud and the Container Engine but I cannot get my VM to access my kubernetes "service"
My scenario: I have a Jenkins VM deployed in the Compute engine (everthing is default network). Then I deployed a Container Engine cluster and I started up a docker registry (with a pod file) and used a "service" in front of the pod. I can access the docker registry just fine from all cluster hosts and the master just fine. I also tried to create a external load balancer so that the docker registry is public. All works fine.
However, I don't want the registry to be public. I just want my Jenkins VM to be able to access the docker registry "service" in the Container cluster. But it does not work. It looks like they are on a different subnet. Jenkins has the IP 10.240.126.57 and the docker-registry-service in the Container cluster has 10.131.249.127.
What is the best way to achieve this? I played with "Routes" but for some reason I'm incapable to get this to work. Sidenote: all VMs can ping all the container and vice versa, but I just cannot access "Container engine service IPs".
Try:
gcloud compute routes create svc-fwd-1 --destination-range 10.131.240.0/20 --next-hop-instance k8s-<clusterName>-node-1 --next-hop-instance-zone <zone>
(replacing with the name of your cluster, and with the zone the cluster is in)
GCE VMs are given private IPs in the networks CIDR range (10.240.0.0/16 for the default network).
GKE pods are given private IPs in the cluster's container CIDR range (10.128.0.0/14 in your case).
GKE services are given private IPs in the cluster services CIDR range (10.131.240.0/20 in your case).
GKE creates GCE Routes to get traffic to the right pods, but doesn't add anything to expose Services from outside the cluster.
The GCE Route above will forward any traffic to 10.131.240.0/20 on the default network to node-1 of your cluster. The service proxy running there will take care of getting traffic to the pod that implements the service. If you'd like, you could even add a route for each of your gke nodes to balance traffic among each of the service proxies.
Note: I haven't actually tried this, so let me know if it works! :)