I'm trying to move away from our aging on-prem .NET framework deployment to an Azure multi region architecture (.NET Core Web API), mostly because of latency and scaling requirements (and later, data sovereignty). We are currently hitting about 40-80 requests/sec, mostly basic CRUD activity with JSON as the transport data (avg 1KB, max 3-4 KB per request). Our current stack is NGINX LB (+ModSec WAF) > 4x IIS > MongoDB + SQL.

I've been diving into K8S/Docker and Azure/GCP/AWS and trying to setup some demo projects so I can learn as much as possible. I'm still struggling a bit with some details, would love it if you guys can take a look at my proposed architecture.

I have quite a few questions (after getting very confused by different answers from some lacking, ever-changing documentation):

• Completely managed, serverless container deployment

  • Can't seem to find this solution in Azure, other than ACI (Azure Container Instances) which seems immature, doesn't support autoscaling (AFAIK), no multi-region deployment etc. I tried Google Cloud Run and it seems perfect, yet the Google BigTable pricing is crazy expensive ($470 as base price!) so I had to drop that direction. I also tried AWS ECS Fargate but the documentation was so lacking, full of outdated/wrong information even for simple demo projects, I decided against it also. As I write .NET Core C# and the Azure documentation looks the best out of the 3 big CSPs, I think it is a good choice.

• Azure Front Door (Global Load Balancer)

  • Do I get a single static IP ? if so, how would it serve responses with low latency around the globe? or is it managing the DNS by itself and creates several global instances and works in an LBR (latecy based routing) mode? If it manages the DNS, do I have to move my DNS to Azure?
  • It targets "eu.api.test.com" and "na.api.test.com", assuming I don't want these hostnames to be public, can I somehow associate Azure Front Door with an internal DNS?

• VLAN/Subnets

  • Can I isolate the VLANs this way? One of my concerns is directly exposing the K8S cluster to the internet. I would also like to create another even more sensitive VLAN which holds the data. Is it possible to limit an Azure CosmosDB interface to communicate only with this specific VLAN?
  • When activating Cosmos DB multi-region replication (active-active), do I need to setup a new VLAN that connects both regions? or is it done automatically by Azure?

• Kubernetes and AGIC - Application Gateway Ingress Controller

  • AFAIK, AGIC is just another K8S Service that's hosted inside a Node that communicates with Azure Resource Manager that in turn talks to Application Gateway to let it know which Pods are able to serve back requests. Is it a good practice to host AGIC in another "node pool" than the application node pool?
  • Is it better to a install a completely separate K8S cluster in each region or should I manage all global Nodes from a single region?
  • Is it a good practice to create a separate K8S system Node Pool and application Node Pool?
  • How do I know which configuration (CPU + Memory + Disk speed) is suited for both the system node pool and my application node pool? I'm trying to understand the balance between number of nodes VS single node capacity (in respect to costs).
  • What is the best practice in regards to number of pods per node (my intention is hosting only one app - no other microservices needed), is it wise to setup 1 Pod per Node with 100% resources assigned to it?
  • How to setup autoscaling? Should I scale both Nodes (VM instances) and Pods inside each Node?

Thanks a lot!