If some nodes down in a cluster and the status become NotReady
, can I make it to trigger some action?
I tried to use client-go and curl to watch /api/v1/watch/nodes
and get notifies when nodes status have any
change. How do I get notifies only if nodes status change from Ready
to NotReady
?
As of now, there is no way to query for specific updates. There are two ways you can implement this:
The NodeStatus has a field called lastTransitionTime
which records a change in a field. In your application, you can check every X seconds and then compare last checked time with lastTransitionTime
to determine if there is a change. If your application restarts, you will have to do a one time check at startup of application
{
"type": "Ready",
"status": "True",
"lastHeartbeatTime": "2017-12-27T09:52:19Z",
"lastTransitionTime": "2017-12-26T14:55:49Z",
"reason": "KubeletReady",
"message": "kubelet is posting ready status"
}
Your app written in client-go can maintain a cache and compare the value in cache to the value from watch events. In this case as well, you will have to think of ways to not miss events when your watcher service is down. You can either store that state somewhere or use some other mechanism but it depends on criticality of your use case.
The closest example I can find is in Calico-go, where any changes in network policies are watched for and related changes are done in a calico-etcd so that agents on nodes can implement firewall rules accordingly. Starting on this line, events are handled, of course you will have to navigate the stack further! Code