Multi-tenant microservice design with high memory usage in each tenant

10/21/2019

I'm designing a containerized microservice and want to make it multi-tenant. Deployments are on Kubernetes. Each tenant/app has high memory usage due to several data structures that need to be initialized in-memory during an init call (e.g., POST service/v1/<app_id>/init) and these app-, tenant-specific data structures will be exploited in subsequent REST calls (e.g., GET service/v1/<app_id>/resources/). Assuming each tenant-specific in-memory data structure consumes about 500mb, I'm wondering what the best possible path to take in designing this system, especially taking into account issues such as auto-scaling the services. Currently, I'm evaluating the following approaches:

  1. Server-side sessions: Using something like Python Beaker sessions, I'm thinking of maintaining all tenant-specific data structures / Python objects in sessions. However, I'm not certain how this would work when the containers/pods have to auto-scale since I haven't figured out which sessions to initiate during auto-scaling (e.g., only initialize the last tenant/app that was accessed???)
  2. One container per app/tenant: It will take some orchestration on docker/k8s side (figuring out network & routing rules) but this is another approach I've looked into. So, we design the microservice such that it uses a single docker image, each container only contains code/data structures specific to one tenant/app.
  3. Kubernetes namespaces: Not sure how appropriate it is for this use case. But I can allocate one namespace per app.

I wonder if any SO members have come across a similar problem and figured out a path forward. Ideally, I'd like to make a single container/pod be able to serve multiple tenants/apps, but really want to know how to make sure things don't fall apart during scaling. Looking for some input before I delve into one approach and go down a rabbit hole.

Thanks much for any input.

-- Niranjan
docker
kubernetes
microservices
multi-tenant

0 Answers