Is there a way to disable GPU for CPU pods?

8/23/2021

I am running a GPU server by referring to this document. I have found that GPU is used in DL work with Jupyter notebook by creating a virtual environment of CPU pod on the GPU node as shown below.

Obviously there is no nvidia.com/GPU entry in Limits, Requests, so I don't understand that GPU is used.

Limits:
cpu: 2
memory: 2000Mi
Requests:
cpu: 2
memory: 2000Mi

Is there a way to disable GPU for CPU pods?

Thank you.

-- OH KYOON
gpu
kubernetes

1 Answer

8/23/2021

Based on this topic on github:

This is currently not supported and we don't really have a plan to support it.

But...

you might want to take a look at the CUDA_VISIBLE_DEVICES environment variable that controls what devices a specific CUDA process can see:
https://devblogs.nvidia.com/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/

-- Mikołaj Głodziak
Source: StackOverflow