My ceph cluster is running on AWS with 3 masters 3 workers
configuration. When I do kubectl get nodes
it shows me all the nodes in the ready
state. Is there is any way I can simulate manually to get nodeNotReady
error for a node?.
Not sure what is the purpose to simulate NotReady
kubectl cordon node NODE_NAME
This will add the unschedulable taint to it and prevent new pods from being scheduled there.kubectl drain NODE_NAME
In general you can play with taints and toleration to achieve your goal related to the above and you can much more with those!
Now NotReady status comes from the taint node.kubernetes.io/not-ready Ref Which is set by
In version 1.13, the TaintBasedEvictions feature is promoted to beta and enabled by default, hence the taints are automatically added by the NodeController
Therefore if you want to manually set that taint kubectl taint node NODE_NAME node.kubernetes.io/not-ready=:NoExecute
the NodeController will reset it automatically!
So to absolutely see the NotReady status this is the best way
Lastly, if you want to remove your networking in a particular node then you can taint it like this kubectl taint node NODE_NAME dedicated/not-ready=:NoExecute
If you just want NodeNotReady you can delete the CNI you have installed. kubectl get all -n kube-system
find the DaemonSet of your CNI and delete it or just do a reverse of installing it: kubectl delete -f link_to_your_CNI_yaml
You could also try to overwhelm the node with too many pods (resources). You can also share your main goal so we can adjust the answer.
About the answer from P Ekambaram you could just ssh to a node and then stop the kubelet.
To do that in kops you can just:
ssh -A admin@Node_PublicDNS_name
systemctl stop kubelet
EDIT: Another way is to overload the Node which will cause: System OOM encountered
and that will result in Node NotReady state. This is just one of the ways of how to achieve it: SSH into the Node you want to get into NotReady Install Stress Run stress: stress --cpu 8 --io 4 --hdd 10 --vm 4 --vm-bytes 1024M --timeout 5m
(you can adjust the values of course) Wait till Node crash. After you stop the stress the Node should get back to healthy state automatically.
just stop kebelet service on one of the node that you want to see as NodeNotReady