vsphere cloud provider vcp stopped working after update to kubernetes 1.11.0

7/7/2018

I had vcp working perfectly on kubernetes v1.9.6 . I had the installation done using kubeadm on our vsphere/vcenter environment .

Yesterday i did update from 1.9.6 -> 1.10.4 -> 1.11.0 , but after the update I am not able to put vcp back to work. I tried complete reinstall but still it is not working . It is complaining about VM Not Found as below

Jul  7 05:08:41 localhost journal: E0707 05:08:41.853267       1 datacenter.go:78] Unable to find VM by UUID. VM UUID: 
Jul  7 05:08:41 localhost journal: E0707 05:08:41.853374       1 nodemanager.go:414] Error "No VM found" node info for node "engine01" not found
Jul  7 05:08:41 localhost journal: E0707 05:08:41.853416       1 vsphere_util.go:134] Error while obtaining Kubernetes node nodeVmDetail details. error : No VM found
Jul  7 05:08:41 localhost journal: E0707 05:08:41.853444       1 vsphere.go:1160] Failed to get shared datastore: No VM found
Jul  7 05:08:41 localhost journal: I0707 05:08:41.854301       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-disk", UID:"a7c88c13-813d-11e8-aa8c-0050568166d0", APIVersion:"v1", ResourceVersion:"31538424", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' Failed to provision volume with StorageClass "thin-disk": No VM found
Jul  7 05:08:43 localhost journal: I0707 05:08:43.768259       1 reconciler.go:291] attacherDetacher.AttachVolume started for volume "pvc-a3091746-6a16-11e8-87de-0050568166d0" (UniqueName: "kubernetes.io/vsphere-volume/[3PAR_Datastore06] kubevols/kubernetes-dynamic-pvc-a3091746-6a16-11e8-87de-0050568166d0.vmdk") from node "engine02" 
Jul  7 05:08:43 localhost journal: E0707 05:08:43.785846       1 datacenter.go:78] Unable to find VM by UUID. VM UUID: 
Jul  7 05:08:43 localhost journal: E0707 05:08:43.785913       1 nodemanager.go:282] Error "No VM found" node info for node "engine02" not found
Jul  7 05:08:43 localhost journal: E0707 05:08:43.785938       1 vsphere.go:550] Cannot find node "engine02" in cache. Node not found!!!
Jul  7 05:08:43 localhost journal: E0707 05:08:43.786012       1 attacher.go:80] Error attaching volume "[3PAR_Datastore06] kubevols/kubernetes-dynamic-pvc-a3091746-6a16-11e8-87de-0050568166d0.vmdk" to node "engine02": No VM found

OR

Jul  7 05:31:40 localhost journal: E0707 05:31:40.976785       1 datacenter.go:78] Unable to find VM by UUID. VM UUID: 
Jul  7 05:31:40 localhost journal: E0707 05:31:40.976856       1 nodemanager.go:414] Error "No VM found" node info for node "engine01" not found
Jul  7 05:31:40 localhost journal: E0707 05:31:40.976883       1 vsphere_util.go:134] Error while obtaining Kubernetes node nodeVmDetail details. error : No VM found
Jul  7 05:31:40 localhost journal: E0707 05:31:40.976900       1 vsphere.go:1160] Failed to get shared datastore: No VM found
Jul  7 05:31:40 localhost journal: I0707 05:31:40.977444       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-disk", UID:"a7c88c13-813d-11e8-aa8c-0050568166d0", APIVersion:"v1", ResourceVersion:"31538424", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' Failed to provision volume with StorageClass "thin-disk": No VM found
-- Shahbour
kubernetes
vsphere

0 Answers