GCloud K8s Pod-to-Pod Routing

11/15/2018

Currently, I'm running a Kubernetes cluster on GCloud K8s Engine.

Now I'm running an OpenVPN Server on there to create a network where multiple clients can talk together in a client-to-client fashion. If I'm using just a single VPN server it also already works. The client can connect to the K8s Pod and communicate with other clients or even the server itself.

However now I want to make that a little bit more available and want to have at least two servers which means I create another VPN network, which is relatively simple, by using the same configuration I used in server1 (I just need to adjust the Subnet).

But the tricky part is how can I make it happen that both pods can correctly route the networks?

i.e. I have the VPN networks 172.40.0.0/16 (Pod 1) and 172.41.0.0/16 (Pod 2). Does K8s or GCloud have any way of announcing the VPN network so that the pods will correctly route requests from 172.40.0.0/16 to 172.41.0.0/16

(OpenVPN will have both routes pushed to the client, so either Pod 1 will be the gateway or Pod 2)

I wouldn't bother writing code so that I can correctly communicate with the pods i.e. if I create a GCloud Route with the POD IP as a gateway with the networks would that work?

-- Christian Schmitt
gcloud
kubernetes

1 Answer

11/16/2018

Does K8s or GCloud have any way of announcing the VPN network so that the pods will correctly route requests from 172.40.0.0/16 to 172.41.0.0/16

Kubernetes doesn't have any such mechanisms. However, you could look at BGP peering with Calico as an overlay.

The other option I guess is to create manual routes on both servers that point to each other that way traffic flows both ways. Traffic to the PodCidr is gonna be trickier because it's generally masqueraded with iptables and in a Kubernetes cluster the PodCidr is cluster-wide.

-- Rico
Source: StackOverflow