My Kubernetes cluster uses a replicaSet to run N similar pods. Each pod can only handles one websocket connection du to resource limitations. My Kubernetes uses a nginx ingress controller.
Is there any way to make nginx dispatch only one incoming websocket connection per pod and in case of lack of available pods, refuses the incoming connection ?
I'm not super familiar with the Kubernetes Nginx ingress setup, but assuming it exposes some of the Nginx configuration options for setting up groups of servers, in the server function there's a parameter called max_conns that will let you limit the number of connections to a given server. Assuming there's a mapping in the ingress controller, it should be possible to set max_conns=1 for each server that's getting created and added to the Nginx configuration under the hood.
http://nginx.org/en/docs/http/ngx_http_upstream_module.html#server
Edit: a little cursory research and it looks like this is indeed possible. It looks like you can specify this in a ConfigMap as nginx.org/max-conns according to the master list of parameters here: https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/configmap-and-annotations.md
You could use the readinessProbe with a periodSecond relatively low and, obvously, {success,failure}Threshold set to 1 in order to release or not the Pod as fast as possible.
Basically, you could set up a script or a simple HTTP endpoint that returns a failed status code in case a connection has been established: so the Pod endpoint will be removed from the Service endpoints list and will not be selected by the Ingress Controller.
Just keep in mind that this solution could be affected by race-conditions but it's the most simple one: a better solution could be using a Service Mesh but means additional complexity.