Standalone WebSocket client throught Ingress nginx fails

11/20/2019

In a Hosted Rancher Kubernetes cluster, I have a service that exposes a websocket service (a Spring SockJS server). This service is exposed to the outside thanks an ingress rule:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myIngress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600s"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600s"
    nginx.ingress.kubernetes.io/enable-access-log: "true"
spec:
  rules:
    - http:
        paths:
        - path: /app1/mySvc/
          backend:
            serviceName: mySvc
            servicePort: 80

A web application connects to the web socket service throught an ingress nginx and it works fine. The loaded js scripts is:

    var socket = new SockJS('ws');
    stompClient = Stomp.over(socket);

    stompClient.connect({}, onConnected, onError);

On the contrary, the standalone clients (js or python) do not work as they returns a 400 http error.

For example, here is the request sent by curl and the response from nginx:

curl  --noproxy '*' --include \
     --no-buffer \
     -Lk \
--header "Sec-WebSocket-Key: l3ApADGCNFGSyFbo63yI1A==" \
--header "Sec-WebSocket-Version: 13" \
--header "Host: ingressHost" \
--header "Origin: ingressHost" \
--header "Connection: keep-alive, Upgrade" \
--header "Upgrade: websocket" \
--header "Sec-WebSocket-Extensions: permessage-deflate" \
--header "Sec-WebSocket-Protocol: v10.stomp, v11.stomp, v12.stomp" \
--header "Access-Control-Allow-Credentials: true" \
https://ingressHost/app1/mySvc/ws/websocket


HTTP/2 400
date: Wed, 20 Nov 2019 14:37:36 GMT
content-length: 34
vary: Origin
vary: Access-Control-Request-Method
vary: Access-Control-Request-Headers
access-control-allow-origin: ingressHost
access-control-allow-credentials: true
set-cookie: JSESSIONID=D0BC1540775544E34FFABA17D14C8898; Path=/; HttpOnly
strict-transport-security: max-age=15724800; includeSubDomains

Can "Upgrade" only to "WebSocket".

Why does it work with the browser and not standalone clients ?

Thanks

-- crepu
kubernetes
nginx
nginx-ingress
rancher
websocket

4 Answers

11/20/2019

I had used nginx to create ingress until I discovered traefik. you should test with traefik.

-- Antonio Fernandez
Source: StackOverflow

11/21/2019

I activated trace in Spring application and a call with curl gives:

o.s.w.s.s.s.DefaultHandshakeHandler      : Handshake failed due to invalid Upgrade header: null

So, it seems nginx removes the Upgrade header !

When testing with the browser client, the header is present in Spring Applicaton and it works.

-- crepu
Source: StackOverflow

11/25/2019

When I try with http instead of https, it works.

I do not understand why https removes Upgrade header for standalone clients.

-- crepu
Source: StackOverflow

11/20/2019

The problem doesn't seem to be in the nginx Ingress. The presence of the JSESSIONID cookie most likely indicates that the Spring applications gets the request and sends a response.

A quick search through the Spring's code shows that Can "Upgrade" only to "WebSocket". is returned by AbstractHandshakeHandler.java when the Upgrade header isn't equal to WebSocket (case-insensitive match).

I'd suggest double-checking that the "Upgrade: websocket" header is present when making the call with curl.

Also, this appears to be a similar problem and may apply here as well if the application has several controllers.

And for what it's worth, after replacing ingressHost appropriately I tried the same curl command as in the question against https://echo.websocket.org and a local STOMP server I've implemented some time ago. It worked for both.

You may have already done this but have you tried capturing in the browser the Network traffic to see the request/response exchange, especially the one that returns HTTP 101 Switching Protocols? Then try to exactly replicate the call that the browser makes and that is successful. For example, the STOMP client generates a session id and uses a queue/topic, which are put in the URL path in requests to the server (e.g. /ws/329/dt1hvk2v/websocket). The test request with curl doesn't seem to have them.

-- gears
Source: StackOverflow