Listing unscheduleable kubernetes nodes by taints in operator-sdk/client-go

4/23/2019

I am trying to list all the nodes that are set to unscheduleable in an operator-sdk operator. Generally (pre 1.12) this means they have spec.unscheduleable set. So I tried this:

nodes := &corev1.NodeList{}
opts := &client.ListOptions{}
if err := opts.SetFieldSelector("spec.unschedulable=true"); err != nil {
  reqLogger.Info("Failed to set field selector")
}

Which is erroring:

2019-04-23T10:19:39.761-0700    ERROR   kubebuilder.controller  Reconciler error    {"controller": "node-controller", "request": "/nodename", "error": "Index with name field:spec.unschedulable does not exist"}

I'm confused about this because the field selector works from kubectl:

kubectl get nodes --field-selector="spec.unschedulable=true"

Alongside this issue, I have noticed that after v1.12, the spec.unscheduleable field has been deprecated in favour of TaintNodeByCondition. This further complicates things, because now I really don't think I can use the fieldselector anyone, because I don't believe (unless I'm mistaken?) that you can use fieldselector with the taints anyway.

So, my question is - how can I list all of the tainted/unscheduleable nodes in my cluster efficiently, specifically when using the operator-sdk

UPDATE:

I have managed to solve the fieldselector problem by using a v1meta.ListOptions call, like so:

nodes := &corev1.NodeList{}
opts := &client.ListOptions{
Raw: &metav1.ListOptions{
  FieldSelector: "spec.unschedulable=true",
  },
}

However, I still don't know how to do this with taints, so I've edited the question and will leave it open

-- jaxxstorm
client-go
kubernetes

1 Answer

12/21/2019

According to the Controller Runtime docs, using raw options won't work well with the cache.

Instead you should add the desired fields to the index during controller initialization:

idx := myManager.GetFieldIndexer()
idx.IndexField(&corev1.Node{}, "spec.unschedulable", func(o runtime.Object) []string {
    return []string{fmt.Sprintf("%v", o.(*corev1.Node).Spec.Unschedulable)}
})

I don't know if this can help you with the Operator SDK though.

-- Adam
Source: StackOverflow