Summary I have provisioned kubernetes on several vcenter vms using kubespray and everything there is working perfectly. However I am running into trouble with setting up Vsphere as the cloud provider and dynamically provisioning persistent volumes. I have tried following this guide on setting up Vsphere as a provider. All of the pods seem to launch:
vsphere-cloud-controller-manager-5h8vr 1/1 Running 0 77m
vsphere-csi-controller-0 5/5 Running 0 52m
vsphere-csi-node-9gcn5 3/3 Running 0 23m
vsphere-csi-node-lcmvn 3/3 Running 0 23m
vsphere-csi-node-pgbcj 3/3 Running 0 23m
vsphere-csi-node-rvx4m 3/3 Running 0 23m
However if I describe the pvc I get this error:
Warning ProvisioningFailed 64s (x30 over 52m) persistentvolume-controller Failed to provision volume with StorageClass "fast": Cloud provider not initialized properly
The only error I can find is from the vsphere-csi-nodes:
k logs vsphere-csi-node-pgbcj -c vsphere-csi-node
I0709 18:03:55.515251 1 service.go:88] configured: csi.vsphere.vmware.com with map[mode:node]
time="2020-07-09T18:03:55Z" level=info msg="identity service registered"
time="2020-07-09T18:03:55Z" level=info msg="node service registered"
time="2020-07-09T18:03:55Z" level=info msg=serving endpoint="unix:///csi/csi.sock"
I0709 18:03:57.645866 1 config.go:261] GetCnsconfig called with cfgPath: /etc/cloud/csi-vsphere.conf
I0709 18:03:57.645925 1 config.go:265] Could not stat /etc/cloud/csi-vsphere.conf, reading config params from env
E0709 18:03:57.645968 1 config.go:202] No Virtual Center hosts defined
E0709 18:03:57.645983 1 config.go:269] Failed to get config params from env. Err: No Virtual Center hosts defined
I0709 18:03:57.645995 1 node.go:471] Config file not provided to node daemonset. Assuming non-topology aware cluster.
Looking through "secrets" it looks as though all of that information was propagated correctly.
Pre-reqs in place 1. VM hardware-version >15 2. UUID enabled on all vms 3. Vsphere >6.7
Environment 1. Single Master 2. Three Nodes 3. Host OS: Debian 10.04 64bit 4. Kubespray: latest 5. Kubernetes: 1.18.5 6. VM "hardware version": 17 7. Esxi 7 8. Vcenter 7 9. CNI Weave
Config Files
group_vars/all.yml
cloud_provider: "external"
external_cloud_provider: "vsphere"
vsphere.conf
# Vsphere provider
external_vsphere_vcenter_ip: "serveriphere"
external_vsphere_vcenter_port: "443"
external_vsphere_insecure: "true"
external_vsphere_user: "localadminaccout@vsphere.local"
external_vsphere_password: "indestructiblepasswordhere"
external_vsphere_datacenter: "LAB"
external_vsphere_kubernetes_cluster_id: "k8"
vsphere_cloud_controller_image_tag: "latest"
vsphere_syncer_image_tag: "v1.0.2"
vsphere_csi_attacher_image_tag: "v1.1.1"
vsphere_csi_controller: "v1.0.2"
vsphere_csi_liveness_prove_image_tag: "v1.1.0"
vsphere_csi_provisioner_image_tag: "v1.2.2"
vsphere_csi_node_driver_registrar_image_tag: "v1.1.0"
vsphere_csi_driver_image_tag: "v1.0.2"
vsphere_csi_controller_replicas: 1
vsphere_csi_enabled: true
Storage Class
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
datastore: LAB
diskformat: thin
fsttype: ext4
Persistent Volume Claim
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test
namespace: kube-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi
storageClassName: fast