I'm playing around with GitOps and ArgoCD in Redhat Openshift. My goal is to switch a worker node to an infra node.
I want to do this with descriptive YAML Files, and NOT manually by using the command line (that's easy with kubectl label node ...)
In order to do make the node an infra node, I want to add a label "infra" and take the label "worker" from it. Before, the object looks like this (irrelevant labels omitted):
apiVersion: v1
kind: Node
metadata:
labels:
node-role.kubernetes.io/infra: ""
name: node6.example.com
spec: {}After applying a YAML File, it's supposed to look like that:
apiVersion: v1
kind: Node
metadata:
labels:
node-role.kubernetes.io/worker: ""
name: node6.example.com
spec: {}If I put the latter config in a file, and do "kubectl apply -f ", the node has both infra and worker labels. So adding a label or changing the value of a label is easy, but is there a way to remove a label in an objects metadata by applying a YAML file ?
I've pretty successfully changed a node label in my Kubernetes cluster (created using kubeadm) using kubectl replace and kubectl apply.
Required: If your node configuration was changed manually using imperative command like kubectl label it's required to fix last-applied-configuration annotation using the following command (replace node2 with your node name):
kubectl get node node2 -o yaml | kubectl apply -f -Note: It works in the same way with all types of Kubernetes objects (with slightly different consequences. Always check the results).
Note2: --export argument for kubectl get is deprecated, and it works well without it, but if you use it the last-applied-configuration annotation appears to be much shorter and easier to read.
Without applying existing configuration, the next kubectl apply command will ignore all fields that are not present in the last-applied-configuration annotation.
The following example illustrate that behaviour:
kubectl get node node2 -o yaml | grep node-role
{"apiVersion":"v1","kind":"Node","metadata":{"annotations":{"flannel.alpha.coreos.com/backend-data":"{\"VtepMAC\":\"46:c6:d1:f0:6c:0a\"}","flannel.alpha.coreos.com/backend-type":"vxlan","flannel.alpha.coreos.com/kube-subnet-manager":"true","flannel.alpha.coreos.com/public-ip":"10.156.0.11","kubeadm.alpha.kubernetes.io/cri-socket":"/var/run/dockershim.sock","node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"},"creationTimestamp":null,
"labels":{
"beta.kubernetes.io/arch":"amd64",
"beta.kubernetes.io/os":"linux",
"kubernetes.io/arch":"amd64",
"kubernetes.io/hostname":"node2",
"kubernetes.io/os":"linux",
"node-role.kubernetes.io/worker":""}, # <--- important line
"name":"node2","selfLink":"/api/v1/nodes/node2"},"spec":{"podCIDR":"10.244.2.0/24"},"status":{"daemonEndpoints":{"kubeletEndpoint":{"Port":0}},"nodeInfo":{"architecture":"","bootID":"","containerRuntimeVersion":"","kernelVersion":"","kubeProxyVersion":"","kubeletVersion":"","machineID":"","operatingSystem":"","osImage":"","systemUUID":""}}}
node-role.kubernetes.io/santa: ""
node-role.kubernetes.io/worker: ""Let's check what happened with node-role.kubernetes.io/santa label if I try to replace the worker with infra and remove santa, ( worker is present in the annotation):
kubectl get node node2 -o yaml | sed 's@node-role.kubernetes.io/worker: ""@node-role.kubernetes.io/infra: ""@' | sed 's@node-role.kubernetes.io/santa: ""@@'| kubectl diff -f -
diff -u -N /tmp/LIVE-380689040/v1.Node..node2 /tmp/MERGED-682760879/v1.Node..node2
--- /tmp/LIVE-380689040/v1.Node..node2 2020-04-08 17:20:18.108809972 +0000
+++ /tmp/MERGED-682760879/v1.Node..node2 2020-04-08 17:20:18.120809972 +0000
@@ -18,8 +18,8 @@
kubernetes.io/arch: amd64
kubernetes.io/hostname: node2
kubernetes.io/os: linux
+ node-role.kubernetes.io/infra: "" # <-- created
node-role.kubernetes.io/santa: "" # <-- ignored
- node-role.kubernetes.io/worker: "" # <-- removed
name: node2
resourceVersion: "60973814"
selfLink: /api/v1/nodes/node2
exit status 1After fixing annotation, kubectl apply works pretty well replacing and removing labels:
kubectl get node node2 -o yaml | sed 's@node-role.kubernetes.io/worker: ""@node-role.kubernetes.io/infra: ""@' | sed 's@node-role.kubernetes.io/santa: ""@@'| kubectl diff -f -
diff -u -N /tmp/LIVE-107488917/v1.Node..node2 /tmp/MERGED-924858096/v1.Node..node2
--- /tmp/LIVE-107488917/v1.Node..node2 2020-04-08 18:01:55.776699954 +0000
+++ /tmp/MERGED-924858096/v1.Node..node2 2020-04-08 18:01:55.792699954 +0000
@@ -18,8 +18,7 @@
kubernetes.io/arch: amd64
kubernetes.io/hostname: node2
kubernetes.io/os: linux
- node-role.kubernetes.io/santa: "" # <-- removed
- node-role.kubernetes.io/worker: "" # <-- removed
+ node-role.kubernetes.io/infra: "" # <-- created
name: node2
resourceVersion: "60978298"
selfLink: /api/v1/nodes/node2
exit status 1Here are a few more examples:
# Check the original label ( last filter removes last applied config annotation line)
$ kubectl get node node2 -o yaml | grep node-role | grep -v apiVersion
node-role.kubernetes.io/infra: ""
# Replace the label using kubectl replace syntax
$ kubectl get node node2 -o yaml | sed 's@node-role.kubernetes.io/infra: ""@node-role.kubernetes.io/worker: ""@' | kubectl replace -f -
node/node2 replaced
# check the new state of the label
$ kubectl get node node2 -o yaml | grep node-role | grep -v apiVersion
node-role.kubernetes.io/worker: ""
# Replace the label using kubectl apply syntax
$ kubectl get node node2 -o yaml | sed 's@node-role.kubernetes.io/worker: ""@node-role.kubernetes.io/infra: ""@' | kubectl apply -f -
node/node2 configured
# check the new state of the label
$ kubectl get node node2 -o yaml | grep node-role | grep -v apiVersion
node-role.kubernetes.io/infra: ""
# Remove the label from the node ( for demonstration purpose)
$ kubectl get node node2 -o yaml | sed 's@node-role.kubernetes.io/infra: ""@@' | kubectl apply -f -
node/node2 configured
# check the new state of the label
$ kubectl get node node2 -o yaml | grep node-role | grep -v apiVersion
# empty outputYou may see the following warning when you use kubectl apply -f on the resource created using imperative commands like kubectl create or kubectl expose for the first time:
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl applyIn this case last-applied-configuration annotation will be created with the content of the file used in kubectl apply -f filename.yaml command. It may not contain all parameters and labels that are present in the live object.
Try setting the worker label to false:
node-role.kubernetes.io/worker: "false"Worked for me on OpenShift 4.4.
Edit: This doesn't work. What happened was:
What's funny is that the automated process would not delete the label if it was empty instead of set to false.
you can delete the label with
kubectl label node node6.example.com node-role.kubernetes.io/infra-than you can run the kubectl apply again with the new label. You will be up and running.
I would say it's not possible to do with kubectl apply, at least I tried and couldn't find any informations about that.
As @Petr Kotas mentioned you can always use
kubectl label node node6.example.com node-role.kubernetes.io/infra-But I see you're looking for something else
I want to do this with descriptive YAML Files, and NOT manually by using the command line (that's easy with kubectl label node ...)
So maybe the answer could be to use API clients, for example python? I have found this example here, made by @Prafull Ladha
As already mentioned, correct kubectl example to delete label, but there is no mention of removing labels using API clients. if you want to remove label using the API, then you need to provide a new body with the
labelname: Noneand then patch that body to the node or pod. I am using the kubernetes python client API for example purpose
from pprint import pprint
from kubernetes import client, config
config.load_kube_config()
client.configuration.debug = True
api_instance = client.CoreV1Api()
body = {
"metadata": {
"labels": {
"label-name": None}
}
}
api_response = api_instance.patch_node("minikube", body)
print(api_response)