What is the best way to maximize jupyterhub pods resources assignment

12/22/2019

We have an Openshift environment on our company.
We're trying to maximize the resources between our data scientists using jupyterhub.
Is there an option for assigning more resources dynamicly per demand (if there are free resources avaliable).

-- AS IL
docker
jupyterhub
kubernetes

1 Answer

12/22/2019

You can take a look at setting resource restrictions. (Quite counterintuitive, I agree).

In Kubernetes (and therefore in OpenShift) you can set resource requests and limits.

Resource requests are the minimum a pod is guaranteed from the scheduler on the node it runs on. Resource Limits on the other hand give you the capability to allow your pod to exceed its requested resources up to the specified limit.

What is the difference between not setting resource requests and limits vs setting them?

Setting resource requests and limits

  • scheduler is aware of resources utilized. New resources can be scheduled according to the resources they require.
  • scheduler can rebalance pods across the nodes if a node hits maximum resource utilization
  • pods can not exceed the limits specified
  • pods are guaranteed to get at least the resources specified in requests

NOT setting resource requests and limits

  • scheduler is aware of resources utilized. New resources (e.g. pods) can be scheduled based on a best guess basis. It is not guaranteed that those new pods get minimum resources they require to run stable.
  • scheduler is not able to rebalance resources without at least requests
  • pods can utilize memory / cpu without restrictions
  • pods are not guaranteed any memory / cpu time

How to set up resource requests and limits

In the end it should look something like this

apiVersion: v1
kind: Pod
spec:
  containers:
  - image: openshift/hello-openshift
    name: hello-openshift
    resources:
      requests:
        cpu: 100m 
        memory: 200Mi 
        ephemeral-storage: 1Gi 
      limits:
        cpu: 200m 
        memory: 400Mi 
        ephemeral-storage: 2Gi 

Additional information can be found here

-- ckaserer
Source: StackOverflow