I have a 3 node K8s cluster where all nodes are configured as Master as well as worker. I have set a watcher informer so that my client gets notified when the node is removed.

var sharedInformer = informers.NewSharedInformerFactory(kcv.kubeClient.K8sClient, 0)

// Add watcher for the Node.
kcv.nodeInformer = sharedInformer.Core().V1().Nodes().Informer()
kcv.nodeInformerChan = make(chan struct{})

// Node informer state change handler
kcv.nodeInformer.AddEventHandler(cache.ResourceEventHandlerFuncs {
    // When a new node gets created
    AddFunc: func(obj interface{}) {
        kcv.handleAddNode(obj)
    },

    // When a node gets deleted
    DeleteFunc: func(obj interface{}) {
        kcv.handleDeleteNode(obj)
    },
})

// Start the shared informer.
kcv.sharedInformerChan = make(chan struct{})    
sharedInformer.Start(kcv.sharedInformerChan)

// Start the resource (node, pod and namespace) specific informers
kcv.nodeInformer.Run(kcv.nodeInformerChan)
kcv.podInformer.Run(kcv.podInformerChan)
kcv.nsInformer.Run(kcv.nsInformerChan)

Right now, I am seeing three different behaviors.

• When I "shut down" the one of the master VMs, I do not see any notification coming to the client. I am expecting a notification by virtue of above code.
• After the master VM is indeed shut down and try to run "kubectl get pods -n myns", I do not see any pod list and it throws below error.

 kubectl get pods -n myns
 Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get pods)

• When I bring back the earlier shut downed master VM, the cluster comes back and works normally. I am able to list the pods from all three master nodes.

kubectl version info:

kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.7", GitCommit:"6f482974b76db3f1e0f5d24605a9d1d38fad9a2b", GitTreeState:"clean", BuildDate:"2019-03-25T02:41:57Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.7", GitCommit:"6f482974b76db3f1e0f5d24605a9d1d38fad9a2b", GitTreeState:"clean", BuildDate:"2019-03-25T02:41:57Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

Docker version:

Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5

Cluster info:

kubectl cluster-info
Kubernetes master is running at https://10.30.8.92:6443
coredns is running at https://10.30.8.92:6443/api/v1/namespaces/kube-system/services/coredns:dns/proxy
kubernetes-dashboard is running at https://10.30.8.92:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy