How to configure jupyterlab for dask_kubernetes?

2/22/2020

I'm trying to configure jupyterlab dask extension so that the "New" cluster button will create a KubeCluster instead of the default LocalCluster.

Tried to edit ~/.config/dask/labextension.yml so it will have this content:

kubernetes:
  worker-template:
    metadata:
    spec:
      restartPolicy: Never
      containers:
      - args:
          - dask-worker
          - --nthreads
          - '2'
          - --no-bokeh
          - --memory-limit
          - 12GB
          - --death-timeout
          - '60'
        image: ${JUPYTER_IMAGE_SPEC}
        name: dask-${JUPYTERHUB_USER}
        resources:
          limits:
            cpu: "1.75"
            memory: 12G
          requests:
            cpu: 1
            memory: 12G

labextension:
  factory:
    module: dask_kubernetes
    class: KubeCluster
    args: []
    kwargs: {}

(source: https://github.com/pangeo-data/pangeo-cloud-federation/blob/8f7f4bf9963ef1ed180dd20c952ff1aa8df54ca2/deployments/ocean/image/binder/dask_config.yaml#L37-L42)

Yet creating a new cluster using the notebook button still creates a LocalCluster. What am I missing?

-- user3599803
dask
dask-kubernetes
kubernetes
python

1 Answer

2/23/2020

What you've done above seems fine to me.

Have you restarted JupyterLab? It reads the configuration files at startup time.

-- MRocklin
Source: StackOverflow