Architectural suggestion for Kubernetes cluster with huge number of statefulsets and services

8/30/2019

I need a suggestion on managing thousands of services and statefulsets(1 pod) in a Kubernetes cluster.

Each of the pods needs at least 500mb of memory and these statefulsets are not always up, it will be down for some time & run for some time.

What kind of nodes I should use and what kind of tools I should use here to reduce the billing?

Thank you

-- Imrahamed
google-kubernetes-engine
kubernetes
kubernetes-pod

2 Answers

8/30/2019

The type of nodes you need to use would be based on the type of workload your pods would be performing. https://cloud.google.com/compute/docs/machine-types

You reduce the billing of the cluster by always maximizing resource utilization.

Cluster Autoscaller will help you achieve the same. Increases /Decreases the number of nodes in the cluster based on load. https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler

You can also reduce your billing by using Preemptible VM's or Committed Use Discounts

Preemptible VM's Since all the nodes in the Kubernetes cluster can be replaced by a similar VM. (cattle not pets!) https://cloud.google.com/compute/docs/instances/preemptible

Committed Use Discounts GCP also reduces the cost of the VM's if you are committed to using the VM's for a longer duration https://cloud.google.com/compute/docs/instances/signing-up-committed-use-discounts

-- Tummala Dhanvi
Source: StackOverflow

8/30/2019

You can use https://cloud.google.com/kubernetes-engine/docs/how-to/preemptible-vms, but please note that your application should be designed to handle potential outages caused by rotation of preemptible nodes i.e. application should be fault-tolerant.

Also you can use cluster autoscaler https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler so your cluster size will adapt to the actual demand.

-- Stepan Vrany
Source: StackOverflow