How to expose Kubernetes Deployments as Services using the static external IPv4-Address of the Worker Nodes?

6/20/2019

I've set up a Kubernetes Cluster on a custom OpenStack Platform to which, I don't have any administration access. It is only possible to create Instances and assign Firewall-Rules to them. Each new instance will be automatically provided by a static external IPv4 Address which can be reached globally. This means, that I can't even create OpenStack Routers to my internal network.

So far so good, I've set up a Kubernetes Cluster using kubeadm, CoreDNS and flanel as CNI. The cluster Hardware Setup is as following:

  • Kubernetes-Client and Server-Version: 1.14.3 linux/amd64
  • All Servers run on Fedora Cloud Base 28
  • 1 Kubernetes Master
  • 5 Worker Nodes
  • 6 static external IPv4 addresses (one for each of the nodes)

After the setup, I deployed my required services using deployment-files. Everything works as it should.

My question is now, how I can make the services externally accessible? Since I've no LoadBalancer provided by OpenStack? How is the best approach for this?

I'm asking this question after an estimated amount of four hours of Googling (maybe I'm just bad at it). I tried the suggested approaches from the Documentation, but it stays totally unclear for me, what the concept and the right approach for the task is.

For example I tried to assign external IPs to the Service by using for example

kubectl expose deployment $DEPLOYMENT_NAME \
                          --name=$SERVICE_NAME \
                          --port=$HOST_PORT \
                          --target-port=$TARGET_PORT \
                          --type=NodePort

or this

kubectl patch service $SERVICE_NAME -p '{"spec":{"externalIPs":["<worker_host_ip>"]}}'

Even if the external IP is now assigned, the routing to my destination service is still not routed properly, because as I get it, Kubernetes automatically assigns the hosts and random ports to the Pods (which is the desired behaviour), but with that in mind, every redeployment could crash the assigned IP to service mapping.

After your help and a big "Thank You!" in advance I expect, that I can assign the application ports of the containers, to the static IPv4 of one of the hosts and that Kubernetes automatically knows, that the deployed service will be routed over this specific IP even, if the Pods run on a different worker.

-- Viktor Krimstein
docker
kubernetes
openstack

1 Answer

7/15/2019

After a while of researching, I stumbled upon the MetalLB implementation of a bare metal load-balancing solution for Kubernetes. This helped me to solve the issue described above. However from my point of view, MetalLB should be only used as last chance, since it is not production ready and requires excessive configration using the NGINX Ingress Controller solution to properly map the IPv4 distribution of a cluster.

Anyway a big big thank you to the above gentleman, which were willing to support me and give advice!

-- Viktor Krimstein
Source: StackOverflow