How to partitions space between n pods in kubernetes

5/6/2019

We are using Kubernetes and we need to do "Smart partitioning" of data. We want to split the space between 1 to 1000 between n running Pods, And each pod should know which part of the space is his to handle (for pooling partitioned tasks).

So for example, if we have 1 pod he will handle the whole space from 1-1000.

When we scale out to 3 pods, each of them will get the same share.

Pod 1 - will handle 1-333

Pod 2 - 334-667

Pod 3 667-1000

Right now the best way that we find to handle this issue is to create a Stateful-set, that pooling the number of running pods and his instance number and decide which part of the space he needs to handle. Is there a smarter/built-in way in Kubernetes to partition the space between nodes in this manner?

Service fabric has this feature built-in.

-- Amir Harari
kubernetes
partitioning

1 Answer

5/6/2019

There are NO native tools for scaling at the partition level in K8s yet.

Only custom solutions similar to what you have came up with in your original post.

-- A_Suh
Source: StackOverflow