I have a cluster where several projects are deployed under different namespaces.
nginx-ingress was installed with helm on the cluster, and a new ingress is deployed for every project.
The problem is when adding a new project, i.e creating a new namespace and adding several deployments + ingress service, nginx restarts and drop all connections, disrupting services in all other namespaces.
It seems that even if the nginx-ingress service is in the project namespace, the nginx-ingress pods are shared in the default namespace. Thus I am wondering if there's a way to dedicate nginx-ingress pods to every namespace so changing the config for one namespace doesn't disrupt the whole cluster?
You can deploy a dedicated Ingress controller per namespace (with use of the Helm chart: https://github.com/helm/charts/tree/master/stable/nginx-ingress and custom ingressClass)
controller.ingressClass name of the ingress class to route through this controllerand than use appropriate Ingress annotations to exploit them:
metadata:
name: fooA
annotations:
kubernetes.io/ingress.class: "namespaceA"
metadata:
name: fooB
annotations:
kubernetes.io/ingress.class: "namespaceB"
metadata:
name: fooC
annotations:
kubernetes.io/ingress.class: "namespaceC"So all pods within the namespace namespaceA are using a dedicated namespaceA Ingress controller and won't be affected by restarts of Ingress controllers deployed to the other namespaces (including the default one: ingressClass=nginx).
Just keep in mind that deploying a new Ingress controller may cause a new "physical" Load Balancer to be provisioned (adding extra $$ to your cloud provider monthly bills), so it is worth doing only with the services that are being seriously affected by the connection drops (streaming, WebSockets, etc.), well designed REST services shouldn't be seriously affected by short Ingress restarts.