We're running a 6-node bare-metal K8s cluster (3-nodes control and 3-nodes compute plane). The ingress is deployed and it's working well for path-based routes. Recently we've built a HA-loadbalancing cluster in front of the Ingress in order to achieve high-availability of the Ingress itself.
+---------------+ +---------+ +---------+
| Loadbalancers | --> | Ingress | --> | Jenkins |
+---------------+ +---------+ +---------+
The loadbalancers basically run nginx in reverse proxy mode with floating IP address that has a DNS A-type record with wildcard:
*.cnk8sv.infra A <private-IP>
However in this setup we are unable to apply virtual-host-based routing. In such a mode the Ingress rules don't catch any traffic. I have a suspicion that the nginx in front of the K8s is dropping some information in the HTTP headers and hence the Ingress controllers are unable to do the host-based routing.
This is the configuration of the nginx loadbalancers:
location / {
proxy_pass https://backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
The Ingress object:
spec:
rules:
- host: jenkins.cnk8sv.infra
http:
paths:
- backend:
serviceName: jenkins
servicePort: 8080
path: /
As soon as we remove the host filter in the rule the traffic stars flowing (so yes, the backend is up and running). Any help would be greatly appreciated.
The fix was in changing the X-Forwarded-Proto
value to http
in the nginx config because we were targeting the Ingress at the port 80, i.e. HTTP.