I'm trying to set a new default storage class in our Azure Kubernetes Service. (1.15.10). I've tried a few things but the behavior is strange to me.

I've created a new storage class custom, set it to be the default storage class and then I remove the is-default-class from the default storageclass.

default-storage-class.yml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: custom
parameters:
  cachingmode: ReadOnly
  kind: Managed
  storageaccounttype: Standard_LRS
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Delete
volumeBindingMode: Immediate

and the commands:

# create new storage class "custom"
kubectl apply -f ./default-storage-class.yml
# set storageclass as new default
kubectl patch storageclass custom -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
# remove default storage class from default
kubectl patch storageclass default -p '{"metadata": {"annotations":{"storageclass.beta.kubernetes.io/is-default-class":"false"}}}'

At first, it seems to be working fine:

$kubectl get sc
custom (default)   kubernetes.io/azure-disk   6d23h
default            kubernetes.io/azure-disk   14m

But within a minute, without changing anything:

$kubectl get sc
custom (default)   kubernetes.io/azure-disk   6d23h
default (default)  kubernetes.io/azure-disk   16m

I'm probably missing something here, but no idea what.

If I do a kubectl describe sc default in the minute it hasn't changed back :

storageclass.beta.kubernetes.io/is-default-class=false,storageclass.kubernetes.io/is-default-class=false

And a moment later:

storageclass.beta.kubernetes.io/is-default-class=true,storageclass.kubernetes.io/is-default-class=false