Once pod is scheduled to particular node, it is not moved or shifted to any other node in any case.New pod created on available node.

If you don't have deployment or RC, to manage state(number of pods) of your application, it will be lost forever. But if you are using deploy or other objects who is responsible to maintain desired state, then if node goes down, it detects the changes in current state and then it create new pod to another node(Depending on node capacity).

Absolutely agree with praful above. It is quite challenging to evict the pods from failed node and move them on to another available node in 5 seconds. Practically not possible. You need to monitor the node status, allow grace period to confirm that node in indeed down, then mark the status as unhealthy. Finally move the pods to other active node. You can tweak those node monitor parameters to much less values but the downside is control pane performance would be hit as more connections are made between Kubelet and api server. Suggest you run 2 replicas for each pod so that your app is still be available to serve the user requests

You need to change the kube-controller-manager.conf and update the following parameters: (you can find the file in /etc/kubernetes/manifests)

node-status-update-frequency: 10s
node-monitor-period: 5s
node-monitor-grace-period: 40s
pod-eviction-timeout: 30s

This is what happens when node dies or go into offline mode:

1. The kubelet posts its status to masters by --node-status-update-fequency=10s.
2. Node goes offline
3. kube-controller-manager is monitoring all the nodes by --node-monitor-period=5s
4. kube-controller-manager will see the node is unresponsive and has the grace period --node-monitor-grace-period=40s until it considers node unhealthy. PS: This parameter should be in N x node-status-update-fequency
5. Once the node marked unhealthy, the kube-controller-manager will remove the pods based on --pod-eviction-timeout=5m

Now, if you tweaked the parameter pod-eviction-timeout to say 30 seconds, it will still take total 70 seconds to evict the pod from node The node-status-update-fequency and node-monitor-grace-period time counts in node-monitor-grace-period also. You can tweak these variable as well to further lower down your total node eviction time.