I have a Rancher cluster (v2.4.5) running on custom nodes with the following configuration:
user nginx;
worker_processes 4;
worker_rlimit_nofile 40000;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 8192;
}
http {
upstream rancher_servers {
least_conn;
server <MY_NODE_IP>:443 max_fails=3 fail_timeout=5s;
}
server {
listen 443 ssl http2;
server_name example.com service1.example.com service2.example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://rancher_servers;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_read_timeout 900s;
}
}
}
Firewall rules are OK, I can deploy minor web-apps with stuff running on port 80 only and get redirected automatically to HTTPS. An example of YAML I'm using to deploy stuff is the following:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: www-deployment
labels:
app: www
spec:
replicas: 1
selector:
matchLabels:
app: www
template:
metadata:
labels:
app: www
spec:
containers:
- name: www
image: my-www-image
---
kind: Service
apiVersion: v1
metadata:
name: www-service
spec:
selector:
app: www
ports:
- port: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: www-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: service1.example.com
http:
paths:
- path: /
backend:
serviceName: www-service
servicePort: 80
The problem is when I try to deploy a service that runs on both ports 80 and 443 but, when requested on port 80, automatically redirects to port 443. When that's the case, if I specify the Ingress like below (with port 443), I get a Bad Gateway
response not from the host machine NGINX. I can tell that because my host machine runs nginx/1.18.0
and the response comes from nginx/1.17.10
.
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: www-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: service1.example.com
http:
paths:
- path: /
backend:
serviceName: www-service
servicePort: 443
But then, if I change the configuration above to servicePort: 80
I keep getting ERR_TOO_MANY_REDIRECTS, because it enters an infinite loop of redirecting from anything to https://anything
.
Am I doing anything wrong here? How can I do a workaround to make these things work?
Found it out. Turns out that the only thing I needed to do was to tell the nginx-ingress-controller that I was expecting HTTPS connections. Final YAML for exposing the service is the following:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: www-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: service1.example.com
http:
paths:
- path: /
backend:
serviceName: www-service
servicePort: 443