I started last month in a new company. Where I will be responsible for the infrastructure and the backend of the SAAS.
We currently have one droplet/instance per customer. In the current phase of the company it is a good choice. But in the future when the number of instances grow, it will be difficult to maintain. At the moment there are 150 instances online, each with 1CPU and 1GB memory.
Our customers only use the environments for moments a week, a month or a year. So most of the time, they do nothing. So we want to change that. I am thinking of Kubernetes, Docker Swarm or another tool.
What advice can you give us? Should we make the step to Kubernetes or Docker Swarm, or stay with the droplets / VMs at DigitalOcean, AWS or GCP?
If we move to AWS or GCP our average price will go up from 5$ p/m to above the 10$ p/m.
We want to make the next step to lower the waste of resources but also thinking about the monthly bill. In my mind, it will be better to have 2 our 3 bigger VMs running Kubernetes or Docker Swarm to lower the monthly bill and lower our reserved resources.
What do you think?
If you are serious about scaling, then you should rethink your application architecture. The most expensive part of computing is memory (RAM), so having dedicated memory per-customer will not allow you to scale.
Rather than keeping customers separate by using droplets, you should move this logical separation to the data layer. So, every customer can use the same horizontally-scaled compute servers and databases, but the software separates their data and access based on a User Identifier in the database.
Think for a moment... does Gmail keep RAM around for each specific customer? No, everybody uses the same compute and database, but the software separates their messages from other users. This allows them to scale to huge numbers of customers without assigning per-customer resources.
Here's another couple of examples...
Atlassian used to have exactly what you have. Each JIRA Cloud customer would be assigned their own virtual machine with CPU, RAM and a database. They had to grow their data center to a crazy size, and it was Expensive!
They then embarked on a journey to move to multi-tenancy, first by separating the databases from each customer (and using a common pool of databases), then by moving to shared microservices and eventually they removed all per-customer resources.
See:
Salesforce chose to go multi-tenant from the very beginning. They defined the concept of SaaS and used to call themselves the "cloud" (before Cloud Computing as we know it now). While their systems are sharded to allow scale, multiple customers share the same resources within a shard. The separation of customer data is done at the database-level.
See:
Bottom line: Sure, you can try to optimize around the current architecture by using containers, but if you want to get serious about scale (I'm talking 10x or 100x), then you need to re-think the architecture.