Currently we are using the Azure disk provisioner. As far as I can tell, each new pod must be given a brand new Azure disk to mount its volume to. This means that the number of pods (needing persistence) we can have running in K8 is effectively limited to the number of disks a VM is allowed multiplied by the number of VMs.
I can't believe this is correct - surely there is a way to abstract over the Azure disks so that we can have multiple pods use the same disk for persistence. And without using an overhead like Ceph?
What it appears you are looking for is something called Azure Files, instead of Disks. The main difference is exactly what you are describing to desire: disks mount to a single node, and hence are difficult to access by pods assigned a different node. On the contrary, azurefiles will work cross-node. For more information look at either: