AKS Volume long attach time

3/29/2020

Starting pods takes long time (8-10 mins)in some cases due to volume attach process in AKS

the below is the pod describe:

 Warning  FailedMount             2m16s (x4 over 9m10s)  kubelet, aks-test-32607237-vmss000010  Unable to mount volumes for pod "test-33-0_default()": timeout expired waiting for volumes to attach or mount for pod "default"/"test-33-0". list of unmounted volumes=[test-33-volume]. list of unattached volumes=[test-33-volume default-token-7cghr]

from kubelet logs:

Unable to mount volumes for pod "test-33-0_default()": timeout expired waiting for volumes to attach or mount for pod "default"/"test-33-0". list of 
E0326 18:44:01.304384    4131 pod_workers.go:190] Error syncing pod  ("test-33-0_default()"), skipping: timeout expired waiting for volumes to attach or mount for

the below is my azure disk configs:

Volumes:
  bigashow90-volume:
    Type:         AzureDisk (an Azure Data Disk mount on the host and bind mount to the pod)
    DiskName:     bigashow90-volume
    DiskURI:      /subscriptions/9ff17cbf-a92b-49f6-b3dd-3d746d326f9d/resourceGroups/MC_cloud-staging_incortacloud-stagingcluster_eastus/providers/Microsoft.Compute/disks/bigashow90-volume
    Kind:         Managed
    FSType:       ext4
    CachingMode:  ReadWrite
    ReadOnly:     false
  default-token-7cghr:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-7cghr
    Optional:    false

we are creating the disk using az disk create from snapshot

then attaching the disk using the below:

volumes: [
            {
              name: `${name}-volume`,
              azureDisk: {
                kind: 'Managed',
                diskName: `${name}-volume`,
                diskURI:
                  `/subscriptions/${subscription}/resourceGroups/` +
                  `${resource_group_of_pvcs}/providers/Microsoft.Compute/disks/${name}-volume`
              }

volumeMounts: [
                {
                  name: `${name}-volume`,
                  mountPath: '/home/user'
                }

Please let me know if anyone faced the same issue before or how to overcome this issue it may reaches 8-10 minutes ?

-- Moetazbellah Medhat Samy
azure
kubernetes

0 Answers