Schedule pod on specific node without modifying PodSpec

4/3/2018

As a k8s cluster administrator, I want to specify on which nodes (using labels) pods will be scheduled, but without modifying any PodSpec section.

So, nodeSelector, affinity and taints can't be options. Is there any other solution ?

PS: the reason I can't modify the PodSpec is that deployed applications are available as Helm charts and I don't have hand on those files. Moreover, if I change the PodSpec, it will be lost on next release upgrade.

-- Seboudry
kubernetes

2 Answers

4/3/2018

You can use the PodNodeSelector admission controller for this:

This admission controller has the following behavior:

  1. If the Namespace has an annotation with a key scheduler.kubernetes.io/nodeSelector, use its value as the node selector.
  2. If the namespace lacks such an annotation, use the clusterDefaultNodeSelector defined in the PodNodeSelector plugin configuration file as the node selector.
  3. Evaluate the pod’s node selector against the namespace node selector for conflicts. Conflicts result in rejection.
  4. Evaluate the pod’s node selector against the namespace-specific whitelist defined the plugin configuration file. Conflicts result in rejection.

First of all you will need to enable this admission controller. The way to enable it depends on your environment, but it's done via the parameter kube-apiserver --enable-admission-plugins=PodNodeSelector.

Then create a namespace and annotate it with whatever node label you want all Pods in that namespace to have:

kubectl create ns node-selector-test
kubectl annotate ns node-selector-test \
    scheduler.alpha.kubernetes.io/node-selector=mynodelabel=mynodelabelvalue

To test it you could do something like this:

kubectl run busybox \
    -n node-selector-test -it --restart=Never --attach=false --image=busybox

kubectl get pod busybox -n node-selector-test -o yaml

It should output something like this:

apiVersion: v1
kind: Pod 
metadata:
  name: busybox
  ....
spec:
  ...
  nodeSelector:
    mynodelabel: mynodelabelvalue

Now, unless that label exists on some nodes, this Pod will never be scheduled, so put this label on a node to see it scheduled:

kubectl label node myfavoritenode mynodelabel=mynodelabelvalue
-- Janos Lenart
Source: StackOverflow

4/3/2018

No, there are no other ways to specify on which nodes pod will be scheduled, only labels and selectors.

I think the problem with a Helm is related to that issue.

For now, the only way for you is to change a spec, remove release and deploy a new one with updated specs.

UPD

@Janos Lenart provides a way how to manage scheduling per-namespace. That is a good idea if your releases already are split among namespaces and if you don't want to spawn different pods on different nodes in a single release. Otherwise, you will have to create new releases in new namespaces and in that case I highly recommend you to use selectors in the Pods spec.

-- Anton Kostenko
Source: StackOverflow