Services not accesible from K8s node (MetalLB Layer2)

1/21/2022

I have bare-metal deployed Kubernetes cluster on VMware vSphere private cloud.

Spec:

  • OS: Ubuntu 20.04.3 Cloud Image (Also trying 18.04 and 21.04)
  • Kubernetes: 1.23.1 (Also trying 1.21.8) - depolyed with kubeadm and kube-proxy (with strictARP and ipvs mode)
  • CRI: Docker 20.10.12
  • MetalLB v0.11.0 (Layer 2 mode)
  • Cilium 1.11.1
  • All nodes in 192.168.50.0/24 subnet

When I simply add deployment:\ kubectl create deployment nginx --image k8s.gcr.io/nginx:latest

And exposed it:\ kubectl expose deployment nginx --type=LoadBalancer --port=80

All seems good. Service got ExternalIP from MetalLB pool:\ nginx LoadBalancer 10.96.56.118 198.168.50.152 80:32325/TCP 4s

New ip address published on kube-ipvs0 interface:

kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default\ ...\ inet 198.168.50.152/32 scope global kube-ipvs0\ valid_lft forever preferred_lft forever\ ...

But I cant get access to service from all K8s nodes trought ExternalIP, and also from other LAN, curl returns:\ curl: (7) Failed to connect to 192.168.50.152 port 80: No route to host

ClusterIP works good, I can access it from node.

-- Ilya Pozdnov
bare-metal-server
kube-proxy
kubernetes
metallb
vsphere

1 Answer

1/22/2022

Solution is: Your Virtual Machines and services must be in different subnets. In my case I have 192.168.50.0/24 network and I split it to 2 subnets 192.168.50.0/25 - for virtual machines and 192.168.50.128/25 for MetalLB services.

-- Ilya Pozdnov
Source: StackOverflow