GKE Persistant-disk container randomly dies and won't remount because of unmount pending

11/7/2016

For some reason our google containers sometimes restart which we cannot find the cause of , this is fine i guess if they start up quick again which is sadly not the case.

The issue seems to be that the persistant disk volume isn't unmounted quick enough and when the restarted container tries to mount it, it just get stuck and you have to kill it and start it manually.

Is it possible to configure the containers to wait for unmount before re-mounting persistant disks, what would be the correct solution to tackle this problem?

kubelet gke-xxx-xxx-xx Warning FailedMount 

Unable to mount volumes for pod "xxx-xxx-xxx-hdki0_default(UUID)": 

timeout expired waiting for volumes to attach/mount
for pod "xxx-xxx-xxx-hdki0"/"default".

list of unattached/unmounted volumes=[xxx-volume]
kubelet gke-xxx-xxx-xx

Warning FailedSync Error syncing pod, skipping: 
timeout expired waiting for volumes to attach/mount for pod 
"xxx-xxx-xxx-hdki0"/"default". 

list of unattached/unmounted volumes=[xxx-volume]
-- JazzCat
google-kubernetes-engine
kubelet
volume

0 Answers