New to K8s and facing an implementation dilemma. I need to deploy a K8s cluster for multiple NGINX-PHP websites, each with its own domain. The number of websites hosted can increase/decrease regularly, with hundreds/thousands of them deployed at any given time. I have excluded the PHP part to keep the question simple.

Scenario 1 - vhost and SSL termination at ingress

• Create a new Namespace containing a new Service and a Deployment (NGINX pods) for every new website
• Setup the virtual host on NGINX Ingress and route it to the service responsible for the website based on the hostname

Pros:

• Pod/Container level isolation for each website

Possibly a con?

• Each website requires a new service and a deployment. This can result in hundreds or thousands of services/deployments

Scenario 2 - vhost and SSL termination at pod level

• Create a single NGINX Service and a Deployment (with pods running a single NGINX container)
• Each pod has access to shared configmap and tls-secret which contains NGINX config and tls certs for every virtual host
• Ingress routes all traffic to the NGINX service
• The pod running NGINX serves the website content from the document root for the desired virtual host

Pros:

• Single service and deployment
• State less pods

Cons:

• NGINX instance in each pod shall require a reload to load a new vhost config/ssl certs on addition of a new website, thus resulting in a management nightmare

Which one of above scenario is better suited? Are there any other possible scenarios for given problem?