I have azure Kubernetes cluster with the following node pools:
The Content spot pool and Application spot pool are the 2 spot node pools. Rest of the others are regular node pools. I'm using spot node pools to cut the costing of my azure instances as much as possible.
I have deployed 2 nginx pods on Content spot pool and Application spot pool respectively. I set it up in such way that, If the spot node pools are get evicted then the 1 pod of nginx is shift to Content pool and other nginx pod is shift to Application pool with the proper use of tolerations and affinity.
But to achieve above mentioned scenario, I have to setup regular node pools' node count is to 1. If I set the node count of regular node to 0, then it is unable to move pods.
Is there any solution, where "I don't have to insert, delete, update my node pool architecture but also I'm able to cut the azure costing for VMs?"
Thanks.
AKS version used - 1.18.14