websocket connection failed after establishing https in google ingress controller

5/24/2018

I have deployed an application in kubernetes which is served by Google Ingress Controller (Service as ELB). The application is working fine. But the moment I am applying https related configuration, the https is coming but websocket fails.

Below is the service file and configmap

for http:

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
  annotations:
    # Enable PROXY protocol
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
    # Increase the ELB idle timeout to avoid issues with WebSockets or Server-Sent Events.
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
spec:
  type: LoadBalancer
  selector:
    app: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: https

---------------------------------------------------------------------------------------------------


kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
data:
  use-proxy-protocol: "true"

for https:

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-east-1:2xxxxxxxxxxxxxxxxxxx56:certificate/3fxxxxxxxxxxxxxxxxxxxxxxxxxx80" 
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https" 
    # Increase the ELB idle timeout to avoid issues with WebSockets or Server-Sent Events.
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
spec:
  type: LoadBalancer
  selector:
    app: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: http

------------------------------------------------------------------------------------------

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
data:
  use-proxy-protocol: "false"

Am I missing any annotations or data in configmap ? Pls help me out

-- Nikit Swaraj
amazon-elb
kubernetes
kubernetes-ingress
nginx
websocket

1 Answer

5/24/2018

I think the problem is the annotation:

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"

The backend-protocol in ELBs must be TCP for websocket connections.

Also, I see you're using Nginx Ingress Controller, maybe you want to set these variables in the config

proxy-read-timeout: "3600"
proxy-send-timeout: "3600"

To avoid connection closings.

-- Ignacio Millán
Source: StackOverflow