I have a Google Kubernetes Engine cluster that runs pods of several environments (dev, pre, pro). In order to reduce costs I would like to stop some nodes during the night, and keep only the necessary to run production pods. Then in the morning I want to upscale the nodes and start the dev/pre pods.
Peak Hours
Node1 -> ProdPod1 DevPod1
Node2 -> ProdPod2 PrePod1
.
.
NodeN -> DevPod2 ProdPod3Valley Hours
Node1 -> ProdPod1 ProdPod2 ProdPod3Does GKE provide some way of programming this downscaling/upscaling of nodes and pods? If not how could it be done?
I would also add that if you want to save money, and if your dev pods can tolerate some downtime, you'd want to look at using preemptible or spot instances. Some good insights here
The thing is, if you're arbitrarily killing nodes, while Kubernetes was requested to run (and is running) a specific workload utilizing specific resources, how do you expect it to know what pods to kill?
(You can address this with pod priorities, but I wouldn't recommend that).
What you want to look at is the Cluster Autoscaler.
Just make sure to kill the dev/test workloads at night, and the auto-scaler will take care of the rest.
I'd get into more detail, but this sounds extremely like a premature optimization, so I don't recommend taking the path you're going to take unless the amount of money you'll be saving is significant, i.e., worth more than your time ;-)
Hope this helps!