We have setup Kubernetes on bare metal RHEL machine. Everything is working fine until we have noticed a random restart of all of the pods in the cluster. As this is our production environment running in kubernetes cluster (kubectl git version 1.13.1), we cannot afford downtime of our services. But we are experiencing random restarts which causes few minutes of downtime of our services. Analyzing kubectl events, the reason it displays is sandbox changed. I manually logged in to all worker nodes and checked the docker containers. All containers on worker nodes restarted together. I am unable to come to any conclusion of this. This automatic restarts is happening on weekly basis. Any help here would be highly appreciated.
Following is the version:
Kubernetes v1.13.1
Docker 18.06.1-ce