K
Q

Question

How to give a Kubernetes service a static address on bare metal across multiple subnets?

3/3/2017

I'm working on a Kubernetes deployment with the following properties:

The cluster is run on our own bare metal machines (not in a cloud provider like GCE or AWS)
The worker machines span multiple subnets

For security reasons, we would like to do the following:

Pin certain services to only be scheduled on machines in particular subnets (this is easy, and we've already figured out how to do this)
For services which need to accept communication from services running in other subnets, somehow fix the address (either IP, port, or both) of that service. Then, statically configure ACLs on our routers to only allow these communications.

I have seen a couple of approaches suggested, but as far as I can tell, they all fall short in some way. They are:

Use a LoadBalancer. This won't work because LoadBalancers currently only support GCE or AWS load balancers
Use NodePorts. This won't work because, as far as I can tell, there's no way to instruct Kubernetes to only have a particular subset of machines (in our case, machines in the same subnet as the service) listen on that port and forward traffic to the service (the reason that having all machines in all subnets listen and forward traffic isn't viable is that we can't know ahead of time what IP address/port the service instances will be scheduled at, and so can't statically configure ACLs on our routers).
Use External IPs. This is probably the closest to being viable, but would require that the number of services in a given subnet which need to accept connections from services outside of that subnet is no greater than the number of IPs in the subnet.

Is there some other way of solving this problem that I'm missing? Writing custom software to solve this is an option, but one that we'd like to avoid if possible.

-- joshlf

kubernetes

load-balancing

networking

static-ip-address

1 Answer

3/4/2017

I would use NodePort with fixed external port for services, then use ACLs to limit the access to the nodes that I want, IE in particular subnet.

Example for fixed NodePort:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  type: NodePort
  ports:
    - port: 80
      nodePort: 30080
      name: http
    - port: 443
      nodePort: 30443
      name: https
  selector:
    name: nginx

-- Farhad Farahi

Source: StackOverflow

KQ

How to give a Kubernetes service a static address on bare metal across multiple subnets?

Similar Questions

1 Answer

K
Q