I have 3 cpu nodes and 1 gpu node on my cluster. As a default, the projects should deploy to the cpu nodes and only when specified to the gpu nodes. I added defaultNodeSelector: "graphic=cpu" to the projectConfig section in master-config.yaml to achieve this purpose.
The problem now when i create a deploymentConfig and assign it to the gpu node using node selector, I get the error pod node label selector conflicts with its project node label selector.
I tried removing the defaultNodeSelector line and added nodeSelectorLabelBlacklist: - graphic=gpu to the PodNodeConstraints section in master-config.yaml, but this did not work at all. Despite this config the pods of the default projects got scheduled to the gpu node :/
Does any body has an Idea how to achieve such stuff?
You have set here a cluster-wide configuration which cannot be overridden at the Pod level.
Probably, the simplest fix in this instance, assuming you have other projects that want to leverage the cluster-wide config is to edit the namespace for this particular project and set the openshift.io/node-selector: "".
For each workload in this project then set the desired node as appropriate in each deployment or deploymentconfig. That should fix the issue for you.
Perhaps, if this is not a one off case for you, take a look at other scheduling features such as taints and tolerations or pod affinity