I have a Kubernetes cluster with an external load balancer on a self hosted server running NGINX. I tried to activate the proxy_protocol
in order to get the real_ip
of clients but NGINX logs are
2020/05/11 14:57:54 [error] 29614#29614: *1325 broken header: "▒▒▒▒▒▒▒Ωߑa"5▒li<c▒*▒ ▒▒▒s▒ ▒6▒▒▒▒▒X▒▒o▒▒▒E▒▒i▒{ ▒/▒0▒+▒,̨̩▒▒ ▒▒
▒▒/5▒" while reading PROXY protocol, client: 51.178.168.233, server: 0.0.0.0:443
Here is my NGINX configuration file:
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
stream {
upstream rancher_servers_http {
least_conn;
server <IP_NODE_1>:80 max_fails=3 fail_timeout=5s;
server <IP_NODE_2>:80 max_fails=3 fail_timeout=5s;
server <IP_NODE_3>:80 max_fails=3 fail_timeout=5s;
}
server {
listen 80;
proxy_protocol on;
proxy_pass rancher_servers_http;
}
upstream rancher_servers_https {
least_conn;
server <IP_NODE_1>:443 max_fails=3 fail_timeout=5s;
server <IP_NODE_2>:443 max_fails=3 fail_timeout=5s;
server <IP_NODE_3>:443 max_fails=3 fail_timeout=5s;
}
server {
listen 443 ssl proxy_protocol;
ssl_certificate /certs/fullchain.pem;
ssl_certificate_key /certs/privkey.pem;
proxy_pass rancher_servers_https;
proxy_protocol on;
}
}
Here is my configmap
for the ingress-controller:
apiVersion: v1
data:
compute-full-forwarded-for: "true"
proxy-body-size: 500M
proxy-protocol: "true"
use-forwarded-headers: "true"
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: '{"apiVersion":"v1","data":null,"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app":"ingress-nginx"},"name":"nginx-configuration","namespace":"ingress-nginx"}}'
creationTimestamp: "2019-12-09T13:26:59Z"
labels:
app: ingress-nginx
name: nginx-configuration
namespace: ingress-nginx
Everything was working fine before I add the proxy_protocol
directive but now I got all these broken headers
errors and I can't reach any services behind ingresses without getting a connection reset
error.
What could be wrong with my config ?
Should I use an http reverse proxy instead of a tcp reverse proxy ?
Thank you.
Edit:
I should also say that I doesn't have any service of type LoadBalancer
in my cluster. Should I have one ? I'm thinking of Metallb but I'm not sure what it will add to my configuration as I'm already load balancing to nodes with nginx.