K
Q

How do I create a policy to run a container on every node, except the master unless there is only one node?

1/21/2019

In the Kubernetes Book, it says that it's poor form to run pods on the master node.

Following this advice, I'd like to create a policy that runs a pod on all nodes, except the master if there are more than one nodes. However, to simplify testing and work in single-node environments, I'd also like to run my pod on the master node if there is just a single node in the entire system.

I've been looking around, and can't figure out how to express this policy. I see that DaemonSets have affinities and anti-affinities. I considered labeling the master node and adding an anti-affinity for that label. However, I didn't see how to require that at least a single pod would always come up (to ensure that things worked for single-node environment). Please let me know if I'm misunderstanding something. Thanks!

-- Behram Mistree
kubernetes

Similar Questions

Ingress multiple paths vs multiple ingresses
When to favor Initializers vs Mutating Webhooks
Can I install monocular helm standalone?
Unable to create route rule with kubernete nginx ingress controller
Passing multiple env files in docker run command
Git sync without commit
Istio return "RBAC: access denied" with wildcard path

1 Answer

1/22/2019

How about something like this:

  1. During node provisioning, assign a particular label to each node that should run the job. In a single node cluster, this would be the master. In a multi-node environment, it would be every node except the master(s).
  2. Create a deamonset that has tolerations for any nodes
tolerations:
    - key: node-role.kubernetes.io/master
      effect: NoSchedule
  1. As described in that doc you linked, use .spec.template.spec.nodeSelector to select only nodes with your special label. (node selector docs).

How you assign the special label to nodes is probably a fairly manual process heavily dependent on how you are actually deploying your clusters, but that is the general plan I would follow.

EDIT: Or I believe it may be simplest to just remove the master node taint from your single-node cluster. I believe most simple distributions like minikube will come this way by default.

-- captncraig
Source: StackOverflow