Installing Jupyterhub hub pod is failing with no available volume zone

11/9/2019

When trying to install Jupyterhub on Kubernetes (EKS) I am getting below error in the Hub pod. This is output of describe pod. There was similar issue reported and i tried the solution but it didn't work.

Warning  FailedScheduling  52s (x2 over 52s)  default-scheduler  0/3 nodes are available: 1 Insufficient cpu, 2 node(s) had no available volume zone.

This is my pvc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
  annotations:
    volume.alpha.kubernetes.io/storage-class: default
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2  
allowedTopologies:
- matchLabelExpressions:
  - key: failure-domain.beta.kubernetes.io/zone
    values:
    - us-east-1a
    - us-east-1b
    - us-east-1c    

Source: jupyterhub/templates/hub/pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: hub-db-dir
  annotations:
    volume.alpha.kubernetes.io/storage-class: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: standard

Please let me know if I am missing something here.

-- Mohan
aws-eks
jupyterhub
kubernetes

1 Answer

11/13/2019

According to AWS documentation, an EBS volume and the instance to which it attaches must be in the same Availability Zone. (Source)

In that case, the solution is using only one AZ.

Kubernetes itself supports many other storage backends that could be used zone independently, but of course with different properties (like performance, pricing, cloud provider support, ...). For example there is AWS EFS that can be used in any AZ within an AWS region but with its own tradeoffs (e.g. kubernetes-incubator/external-storage#1030).

This is a know issue reported here.

-- mWatney
Source: StackOverflow