I have a big aks production cluster of 2 nodepool with 50 nodes each, I also have around 15 kinds of microservices with different CPU request and usage.
after research, I found that one of the microservices can be only 2 or 3 instances on the same node, because of the special CPU usage this microservice head.
there is any solution to do this operation in kubernetes? I know I can limit only one of a kind in the same node with nodeAffinity, but I want 2 or 3 on the same node.
I found this alpha option: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/, but it's alpha and it limits the check only to pod from the same namespace, and my microservice in few namespaces, but with the same behavioral
As mentioned, writing a custom scheduler is your best bet for achieving what you want. You can follow a tutorial like this for implementing a scheduler, and you can deploy it in addition to the default scheduler as explained here.
You can specify for each Pod which scheduler to use. For example, you can configure your custom scheduler to be used only for this specific microservice and use the default scheduler for all other workloads.
No, the topology constraints system you found is the closest that exists for this. You’ll need to write your own scheduler for something this specific.