Why multi-region Kubernetes deployments are not recommended?

10/15/2018

Kubernetes documentation is saying that multi-zone clusters are supported, but not the multi-region ones. At the same time Kubernetes has support for both failure-domain/zone and failure-domain/region.

What are the downsides of having my Kubernetes clusters to be multi-zone and multi-region at the same time? Is it only latency and if so what are the required latency numbers for it to be reliable?

On a plus side I see service discovery and being able to deploy applications across multiple regions without and extra tooling on top of it.

I know there's federation v1 and v2 being worked on but it seems to be adding a lot of complexity and v2 is far from being production ready.

-- Maklaus
kubernetes

1 Answer

10/15/2018

This is speculative, but it's informed speculation, so hopefully that means it'll still be helpful

Let's take two things that kubernetes does and extrapolate them into a multi-region cluster:

  • load balancer membership -- at least on AWS, there is no mechanism for adding members of a different region to a load balancer, meaning type: LoadBalancer could not assign all Pods to the Service
  • persistent volume attachment -- similarly on AWS, there is no mechanism for attaching EBS volumes across even availability zones, to say nothing of across regions

For each of those, one will absolutely be able to find "yes, but!" scenarios to demonstrate a situation where these restrictions won't matter. However, since kubernetes is trying to solve for the general case, in a cloud-agnostic way, that's my strong suspicion why they would recommend against even trying a multi-region cluster -- regardless of whether it happens to work for your situation right now.

-- mdaniel
Source: StackOverflow