Advantages to containerized Kubernetes master processes

8/21/2015

Both the Kubernetes HA guide and the From Scratch guide recommend running Etcd, kube-apiserver, kube-controller-manager, and kube-scheduler in containers. The idea of self-hosting Kubernetes on Kubernetes goes back quite a while (see PR 167 on K8s github and issues/PRs linked there), but I haven't found a discussion about why this approach is so beneficial that it should be the 'recommended' way. Here are the benefits and drawbacks as I see them currently:

Benefits:

  • Potentially easy upgrade path to just update manifests and have kubelet pull new images.
  • "Container advantages": binary environment and the host environment separate, leverage others' existing images, etc.
  • Follows the whole Kubernetes pattern, so 'fits the brain' once you are using that pattern extensively.

Drawbacks:

  • Increased installation/configuration complexity in some cases. For example, if your Etcd cluster is separate from your Kubernetes nodes, you now have to install Docker (with possible storage changes depending on Linux distro), kubelet, and Etcd. Without using containerized Etcd, you just have that one binary to install.
  • Increased complexity at run time: With more moving parts, any bug in Docker or kubelet may be able to render critical components non-functional.

I'm new to Kubernetes (and containers) and feel like I might be missing advantages (or underestimating their value) when compared to the extra complexity it introduces. But I also have to choose once way to try. Why are containerized master components the recommended way to run Kubernetes despite the extra complexity?

-- rwehner
kubernetes

1 Answer

9/18/2015

The biggest benefit is streamlined setup for most people. Running a few docker run commands is way easier than downloading binaries, unpacking, fine-tuning init scripts (which are different on every distro), running a supervisor, etc. We have a pretty good process manager - relying on that is powerful.

We also don't recommend sharing etcd, so if you're doing that you are already off the beaten path.

Overall, containerized components are vastly simpler than the alternatives for most people.

-- Tim Hockin
Source: StackOverflow