Jupyterhub On Kubernetes with GPU Worker Nodes

3/19/2020

I have currently facing two-issue with JuypterHub.

First, I have a deployed Juypterhub on IBM kubernetes Services. I have a bare metal in my cluster with one GPU (Nvidia K80). When running Juypterhub and I check for how much GPU is being consumed (I run nvidia-smi command) it tells me that it is 94% consumed though I am not running any workload on that can consume that many resources. Is there a way to resolve this.

Second, is there a way we can have multi-users to use the same GPU on the node? I mean can I scale JuypterHub to use the one GPU resources.

-- Kunal Malhotra
jupyterhub
kubernetes

1 Answer

4/2/2020

I am not sure about the first issue, will need more details to answer it. Answer for your second question is yes. There are multiple way you can do this. By combination of overriding kubespawner, limiting resource guarantee so you can spin up multiple pods on GPU and setting taints and tolerations for choosing GPU nodes.

-- n3o-B
Source: StackOverflow