K
Q

loadbalancing for kubernetes in non-cloud environment

March 14, 2019

I see that kubernetes can use 

ClusterIP
and 
NodePort
and 
LoadBalancing
. For loadbalancing it requires cloud. If I do not have cloud provider how can I loadbalance traffic between nodes?! I know that HAProxy can loadbalance but I think this cloud loadbalancer is different from simple HAProxy

and I want to know what is different between HAProxy and IngressController such as HAProxy and Nginx

I want a loadbalancer to loadbalance traffic between my worker nodes. A service loadbalance traffic between pods.I think ingress controller is layer 7 loadbalancer. I want loadbalancing between my nodes

-- yasin lachini
kubernetes
cloud
load-balancing
haproxy

Similar Questions

How do I force delete kubernetes pods?
What is the purpose of a kubernetes deployment pod selector?
Kubernetes - how to download a PersistentVolume's content
Where is kube-apiserver located
kubernetes label name 63 character limit

3 Answers

June 21, 2020

I see that kubernetes can use ClusterIP and NodePort and LoadBalancing. For loadbalancing it requires cloud. If I do not have cloud provider how can I loadbalance traffic between nodes?!

The easiest way, as you probably know, would be to set the 

Service
to type 
NodePort
, this signals to 
kube-proxy
 to listen to a random port in the default range of 
30000-32767
on every node. Under the hood this random port will be mapped (port-forwarded) to the 
Service
port.

You can now send traffic, let's say to the random port of 

30001
, to any of the nodes and you'll be load balanced internally between the Pods. If you now spin up a e.g. VM in the same network as the nodes or in a network that could reach the nodes and setup load balancing across 
node-{a,b,c}:30001
.

You could, although not recommeneded because of many good reasons, basically just send traffic to one of the nodes (

node-a:30001
) in a multi node cluster and the traffic would still be load balanced internally. This is possible due to the fact that all instances of 
kube-proxy
knows where all the Pods (or 
Endpoints
in the context of a 
Service
) are located at any given time.

Note that the 

kube-proxy
and 
iptables
(may vary!) is the components that implements the 
Service
object in all cases besides when the type is 
LoadBalancer
. 
LoadBalancer
requests will be dispatched to either the built-in or external cloud controller manager.

The 

Ingress
object exists to add L7 logic in-front of one or more 
Service
's, but as you've seen the 
Ingress
is worthless if there's no Ingress Controller implementing it. HAProxy and Nginx Ingress controllers would more or less do the same thing for you but not solving your problem short-term. Yes, you'll have load balancing but not in the way you might think.

If you do not have any form of (private/public) cloud with k8s integration backing your k8s cluster up the Nginx and HAProxy Ingress controllers would only be another 

Service
running in your cluster. You would of course be able to do smart things like proxying, URL routing, hostname matching etc.

One of the questions to answer if you're in a non-cloud provider environment (e.g. bare metal only) is: How do i get an IP-address in the 

EXTERNAL-IP
field of a 
Service
of type 
LoadBalancer
? Note that i'm assuming the output of the 
kubectl get service
command. One good answer is, as already stated in the comments here: MetalLB.

MetalLB will give you automation in regards to configuring the external IP(s) of your 

Service
of type 
LoadBalancer
. But you could also configure the 
externalIPs
 field of the 
Service
object manually and set it to an IP address that would make sense in your environment. Thanks @danielrubambura for pointing this out!

Also see this page over at the official Nginx controller documentation that could shed some light on how and why to use MetalLB in some circumstances.

I'm leaving the comparison between Nginx and HAProxy controllers since i don't think that is important in this case. In the end they'll give you Nginx or HAProxy Pods configured as you want through the 

Ingress
object, with e.g. routing to different 
Service
's based on the 
Host
header in the incoming requests.

Hopefully this clears things up a bit!

-- mikejoh
Source: StackOverflow
March 14, 2019

I am facing the same problem here. K8s is made for the cloud in mind, so on premises brings some trouble to setup. On the article below it gives a detailed explanation about this.

https://medium.com/@JockDaRock/metalloadbalancer-kubernetes-on-prem-baremetal-loadbalancing-101455c3ed48

In summary the solutions are to use a NodePort or external Name services. The approach that I will try here is to use metalLB (https://metallb.universe.tf/, https://github.com/google/metallb) .

-- Leandro Donizetti Soares
Source: StackOverflow
March 20, 2019

There should be no need in kubernetes to balance a load between nodes, because for kubernetes a backend is a pod, not a node.

So, you should consider an Ingress Controller, not a load balancer, since Kubernetes core controllers come without some controllers, and IC is one of them, and the ClusterIP type service already does basic load balancing.

Nginx IC is great. So as Istio (different concepts though). Traefik could be an option too. Check different IC options, and get the Ingress Controller concept clear.

-- suren
Source: StackOverflow