I have a simple Node.js chat example which uses socket.io. I have tested this deployment with a docker container locally, and a web client succesfully connects to the websocket (socket.io). The problem arise when I try to deploy this on a Kubernetes cluster with an ingress Controller.
My question is, is there anything special socket.io specifically needs when using an ingress controller?
The error message I get is <<browser name>> Can't estabilish a connection to the server at ws://<<address>>/socket.io/EIO=3&transport=websocket
Below is the simple implementation of the server side
var io = socket(server);
io.on('connection', (socket)=>{
console.log('made socket connection')
socket.on('chat', (message)=>{
message:message.value
})
Below is the client:
var socket = io.connect("http://localhost:3002", {
upgrade: false,
transports: ['websocket'],
secure: True
});
My ingress yaml file looks like this
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ws-ingress
namespace: websocket-ns
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/websocket-services: "ws-service"
nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
nginx.ingress.kubernetes.io/proxy-send-timeout: '3600'
nginx.ingress.kubernetes.io/server-snippet: |
http {
server {
listen 3002;
location = / {
proxy_set_header Upgrade "websocket";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header Connection "upgrade";
proxy_cache_bypass $http_upgrade;
}
upstream nodes {
hash $remote_addr consistent;
server app01:3002;
}
}
spec:
rules:
- host: "$host_address"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: ws-service
port:
number: 80
Service yaml:
apiVersion: v1
kind: Service
metadata:
name: ws-service
namespace: websocket-ns
labels:
app: ws-service
spec:
type: LoadBalancer
selector:
app: ws-app
ports:
- port: 80
protocol: TCP
targetPort: 3002
Server deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws-app
namespace: websocket-ns
labels:
app: ws-app
spec:
replicas: 1
selector:
matchLabels:
app: ws-app
template:
metadata:
labels:
app: ws-app
spec:
containers:
- name: ws-app
image: themuchy/socketexample
ports:
- containerPort: 3002
name: ws-app
This one is working with one
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
certmanager.k8s.io/cluster-issuer: core-prod
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: ws-service
nginx.org/websocket-services: ws-service
name: core-ingress
spec:
rules:
- host: test.io
http:
paths:
- backend:
serviceName: ws-service
servicePort: 80
tls:
- hosts:
- test.io
secretName: core-prod
Nginx itself will upgrade the HTTP request and convert it to WebSocket. You don't have to add any annotation of the same.
Deployment
and service
, YAML
configuration looking good however try removing the annotation server-snippet
. This way traffic flow will be something like
LB > ingress (Connection upgrade) > service > pods