GKE jenkins kubernetes plugin node selector not working

2/18/2022

I am using Jenkins to deploy my Docker images to GKE by using the kubernetes plugin version 1.30.3. It works fine so far. But now I am trying to choose a defined node pool in GKE by adding a node selector but it doesn't work for me. This is the definition of the podTemplate in my Jenkins files:

podTemplate(label: '...', containers: [...],
  volumes: [..],
  nodeSelector: 'cloud.google.com/gke-nodepool: NAME OF THE NODE POOL'
)

Do you have any idea why this isn't working?

I already checked the resulting pod yaml which doesn't include the defined nodeSelector...

-- Stephan
google-kubernetes-engine
jenkins
jenkins-plugins
kubernetes
nodeselector

2 Answers

2/18/2022

The nodeSelector provides a way to constrain pods with particular labels in GKE. All the node pools have labels with the following format: cloud.google.com/gke-nodepool: POOL_NAME. So you can use a specific node pool by using a nodeSelector and the label that you configured as a POOL_NAME like the following example:

 nodeSelector:
    cloud.google.com/gke-nodepool:default-pool

Based on this, you will need to validate if the label was attached to your pod when it was created, you can check this by running this command: kubectl get nodes to get the names of your cluster's nodes, you can consult this guide to know more about how to set constraints to pods.

-- Leo
Source: StackOverflow

2/20/2022

Complementing Leo's answer, one possible reason for the label not being currently attached to your pod is that you manually upgraded or downgraded your node pool to match the version of the control plane, GKE automatically removes any labels you added to individual nodes using kubectl.

To solve this you can try to:

  1. Update your node label while being extra careful with the limitations to execute this action.
  2. Because upgrading a node pool may disrupt workloads running in that node pool; you can create a new node pool with the desired version and migrate the workload, then make sure the label is correctly attached and finally delete the old one.

I recommend you to take a good look at these 2 links: Add a label to a node and Upgrade a node pool for further details and let me know if this answer is helpful.

-- Hector Martinez Rodriguez
Source: StackOverflow