Support public and private networks on Mesos or Kubernetes?

7/4/2016

I am wondering if Mesos or K8s can offer resources from multiple network interfaces? I would like to attach multiple Network Interfaces (public eth0, private eth1) on mesos (or K8s) slave nodes and would like to bind specific applications that I run on Mesos's slave nodes on specific interfaces? does not mesos Or K8s need distinct physical networks like OpenStack has four distinct physical networks?? is there any reference guide or doc?

-- AngryTony
kubernetes
mesos
mesosphere

1 Answer

7/7/2016

On Kubernetes, there is not a fully supported way to do this. I think this is not supported by docker either (https://github.com/docker/docker/issues/1824)

As a work around, you could sort of do it this way:

  • have one interface be the "default network interface" for pods. It is the one you configure Kubelet and docker to use. Most of your pods use this one. They get a PodIP.

  • For "special" pods that need access to the other interface, or to both, use the "hostNet: true" parameter on those pods, and Kubernetes will not put the pod in a network container. These pods

    • can bind to either interface.
    • will not get a "podIP", but use the IPs of whichever interface they use.
    • you will have to manage port conflicts yourself. You may want to use DaemonSet for these pods.
    • you won't get any NetworkPolicy protection between pods with hostNet.
    • all the pods on the same node with hostNet will be able to talk to each other on localhost, so you get less isolation.

This workaround is good if only one or a few applications need "non-default" networking, and those apps are "system applications", managed by the same team that manages the cluster, rather than by a "less trusted" application team. Or if you have a small organization with only a few people running the Kubernetes cluster.

-- Eric Tune
Source: StackOverflow