In a Hosted Rancher Kubernetes cluster, I have a service that exposes a websocket service (a Spring SockJS server). This service is exposed to the outside thanks an ingress rule:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myIngress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600s"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600s"
nginx.ingress.kubernetes.io/enable-access-log: "true"
spec:
rules:
- http:
paths:
- path: /app1/mySvc/
backend:
serviceName: mySvc
servicePort: 80
A web application connects to the web socket service throught an ingress nginx and it works fine. The loaded js scripts is:
var socket = new SockJS('ws');
stompClient = Stomp.over(socket);
stompClient.connect({}, onConnected, onError);
On the contrary, the standalone clients (js or python) do not work as they returns a 400 http error.
For example, here is the request sent by curl and the response from nginx:
curl --noproxy '*' --include \
--no-buffer \
-Lk \
--header "Sec-WebSocket-Key: l3ApADGCNFGSyFbo63yI1A==" \
--header "Sec-WebSocket-Version: 13" \
--header "Host: ingressHost" \
--header "Origin: ingressHost" \
--header "Connection: keep-alive, Upgrade" \
--header "Upgrade: websocket" \
--header "Sec-WebSocket-Extensions: permessage-deflate" \
--header "Sec-WebSocket-Protocol: v10.stomp, v11.stomp, v12.stomp" \
--header "Access-Control-Allow-Credentials: true" \
https://ingressHost/app1/mySvc/ws/websocket
HTTP/2 400
date: Wed, 20 Nov 2019 14:37:36 GMT
content-length: 34
vary: Origin
vary: Access-Control-Request-Method
vary: Access-Control-Request-Headers
access-control-allow-origin: ingressHost
access-control-allow-credentials: true
set-cookie: JSESSIONID=D0BC1540775544E34FFABA17D14C8898; Path=/; HttpOnly
strict-transport-security: max-age=15724800; includeSubDomains
Can "Upgrade" only to "WebSocket".
Why does it work with the browser and not standalone clients ?
Thanks
I had used nginx to create ingress until I discovered traefik. you should test with traefik.
I activated trace in Spring application and a call with curl gives:
o.s.w.s.s.s.DefaultHandshakeHandler : Handshake failed due to invalid Upgrade header: null
So, it seems nginx removes the Upgrade header !
When testing with the browser client, the header is present in Spring Applicaton and it works.
When I try with http instead of https, it works.
I do not understand why https removes Upgrade header for standalone clients.
The problem doesn't seem to be in the nginx Ingress. The presence of the JSESSIONID cookie most likely indicates that the Spring applications gets the request and sends a response.
A quick search through the Spring's code shows that Can "Upgrade" only to "WebSocket".
is returned by AbstractHandshakeHandler.java when the Upgrade
header isn't equal to WebSocket
(case-insensitive match).
I'd suggest double-checking that the "Upgrade: websocket"
header is present when making the call with curl
.
Also, this appears to be a similar problem and may apply here as well if the application has several controllers.
And for what it's worth, after replacing ingressHost
appropriately I tried the same curl
command as in the question against https://echo.websocket.org
and a local STOMP server I've implemented some time ago. It worked for both.
You may have already done this but have you tried capturing in the browser the Network traffic to see the request/response exchange, especially the one that returns HTTP 101 Switching Protocols
? Then try to exactly replicate the call that the browser makes and that is successful. For example, the STOMP client generates a session id and uses a queue/topic, which are put in the URL path in requests to the server (e.g. /ws/329/dt1hvk2v/websocket
). The test request with curl
doesn't seem to have them.