I would like to run multiple Kubernetes services and use the externalIPs field for those services to bind a specific service to a specific IP.
I have one VM which has three interfaces:
I've already added iproute2 tables/routes/rules for the interfaces 2 and 3, which take care that the traffic is back-routed via the correct interface.
As long is kubelet/kube-proxy is not running, everything is working as expected. (e.g. running nc to serve some data.)
As soon a kubelet/kube-proxy is started some (and I don't know which) iptables configuration is created, which drops the packages. (At least this is what it looks like in tcpdump.)
If I run only one IP on the node, everything works as expected - so I'm assuming the issue is the second IP and some kind of routing.
Here is the iptables config pre and post starting the kubelet service. I've anonymised the file and removed stuff which is clearly unrelated - if I've removed to much, pleas let me know.
Does anybody run a similar setup? How does one need to configure kube-proxy and/or the OS to setup this kind of network? Any ideas where to proceed for debugging?
I'm running Kubernetes 1.6.4 on CentOS7.
Kube-proxy trying to manage all interfaces which it has, and, of course, forcing some rules (include filtering) for provide a service.
If you really want to use multiple interfaces on your servers and save custom forwarding rules between interfaces in the same time, you can bind all your components to the internal interface (eth0 in you case) and manage all other interfaces manually as you want.
For set one interface, you should you that CLI args:
kubelet daemon - --addresskube-proxy daemon - --bind-addresskube-api daemon - --bind-address.But please keep in mind, you will need to use that interfaces for all intercommunication inside a cluster and some flags, like HostNetwork will also provide you only that interface.