I'm working on a SaaS app that will be running in Kubernetes. We're using a Helm chart that deploys all the components into the cluster (for simplicity's sake let's assume it's a frontend service, a backend and a database). App architecture is multi-tenant (we have a single instance of each service that are being shared by all tenants) and we would like to keep it that way. What I'm currently struggling with and would like to ask for advice/best practice on is how does one go about automating the provisioning of custom sub-domains for the tenants?
Imagine the app is hosted at exampleapp.com
. A brand new customer comes and registers a new organisation some-company
. At that moment, in addition to creating new tenant in the system, I would also like to provision a new subdomain some-company.exampleapp.com
. I would like this provisioning to be done automatically and not require any manual intervention.
exampleapp.com
) domain registrar/nameserver provider fit into the solution? Does it have to provide an API for dynamic DNS record creation/modification?I appreciate that the questions I'm asking are quite broad so I'm not expecting anything more than a high-level conceptual answer or pointers to some services/libraries/tools that might help me achieve this.
Note: Since this is more of a theoretical question, I'll give you some points from a Kubernetes Engineer, I divided your question in blocks to ease the understanding.
Question 1:
Imagine the app is hosted at
exampleapp.com
. A brand new customer comes and registers a new organisationsome-company
. At that moment, in addition to creating new tenant in the system, I would also like to provision a new subdomainsome-company.exampleapp.com
. I would like this provisioning to be done automatically and not require any manual intervention.
Suggestion:
kubectl patch
is the simpler solution from my viewpoint.For this approach I suggest installing the Nginx Ingress Controller for it's versatility.
Here is an Example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: <ingress-name>
spec:
rules:
- host: client1.exampleapp.com
http:
paths:
- path: /client1
backend:
serviceName: <main-service>
servicePort: <main-service-port>
- host: client2.exampleapp.com
http:
paths:
- path: /client2
backend:
serviceName: <main-service>
servicePort: <main-service-port>
kubectl patch
on how to add new rules:kubectl patch ingress demo-ingress --type "json" -p '[{"op":"add","path":"/spec/rules/-","value":{"host":"client3.exampleapp.com","http":{"paths":[{"path":"/client3","backend":{"serviceName":"main-service","servicePort":80}}]}}}]'
POC:
$ kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
demo-ingress client1.exampleapp.com,client2.exampleapp.com 192.168.39.39 80 15m
$ kubectl patch ingress demo-ingress --type "json" -p '[{"op":"add","path":"/spec/rules/-","value":{"host":"client3.exampleapp.com","http":{"paths":[{"path":"/client3","backend":{"serviceName":"main-service","servicePort":80}}]}}}]'
ingress.extensions/demo-ingress patched
$ kubectl describe ingress demo-ingress
Rules:
Host Path Backends
---- ---- --------
client1.exampleapp.com
/client1 main-service:80 (<none>)
client2.exampleapp.com
/client2 main-service:80 (<none>)
client3.exampleapp.com
/client3 main-service:80 (<none>)
This rule redirects the traffic incoming from the subdomains to subpaths inside your main app.
Question2 :
How does our (
exampleapp.com
) domain registrar/nameserver provider fit into the solution? Does it have to provide an API for dynamic DNS record creation/modification?
Suggestion:
*.example.app
to the IP of the ingress, I don't believe you need anything more than that, because it redirects all to the ingress and the ingress forwards it internally.Question 3:
If there are some strong arguments why multi-tenancy + Kubernetes don't go along very well, those are welcome as well.
Opinion:
These are my 2 cents to your question, I hope it helps you!