I use k8s to run large compute Jobs (to completion). There are not many jobs nor nodes in my setup so scale is small.
I noticed that the scheduler do not start a pod immediately when a node is available. It takes between 5 to 40 seconds for a pod to be scheduled once node is ready.
Is there a way to make the scheduler more "aggressive"? I cant find a setting for the interval in Default Scheduler custom policy. Is implementing my own scheduler the only way forward? Any tips appreciated!
There is difference between pod scheduling and pod creation. Scheduler only find suitable node and schedule pod to that node but pod creation done by kubelet.
Kubelet polls api-server for desired state and get newly scheduled pod spec and then create pod.
So this process can take time you Specfied in question.
So i dont think writing custom scheduler help here.