kubernetes - resource limits exceeded

3/19/2018

I have a cluster running Kubernetes v1.6.7.

The VMs are :

Distributor ID: Debian Description: Debian GNU/Linux 8.7 (jessie) Release: 8.7 Codename: jessie

Kernel : 3.16.0-4-amd64

We have set cpu limits in deployments, however pods comsume cpu over this limit if they need to. Is there something am I missing ? Like a parameter in kubelet to enable this limit ? I haven't found anything about this problem.

For example, if I create the following deployment :

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: stress namespace: default labels: k8s-app: stress spec: replicas: 1 selector: matchLabels: k8s-app: stress template: metadata: labels: k8s-app: stress spec: containers: - name: stress image: progrium/stress imagePullPolicy: Always args: ["--cpu", "1"] resources: limits: cpu: "500m" memory: "1Gi" requests: cpu: "100m" memory: "512Mi"

On the node, this makes a CPU 100% used whereas it should be 50%.

Thanks for your help.

-- matth3o
cpu
kubernetes
resources

1 Answer

3/19/2018

From a comments to your question, where you can try to run docker container with CPU limit you getting: WARNING: Your kernel does not support CPU cfs period or the cgroup is not mounted. Period discarded. WARNING: Your kernel does not support CPU cfs quota or the cgroup is not mounted. Quota discarded.

That mean your system's kernel build without support of CFS. Here is an issue about it, right about 3.16.0-4-amd64 version.

You need to rebuild or update your kernel.

Here is how to update it from backports:

  • Add mirror with backports: $ sudo vi /etc/apt/sources.list.d/sources.list deb http://http.debian.net/debian jessie-backports main deb-src http://http.debian.net/debian jessie-backports main

  • Update apt: sudo apt-get update

  • Install a kernel from a backport repo:sudo apt-get install -t jessie-backports linux-image-amd64

  • Reboot after instalation.

-- Anton Kostenko
Source: StackOverflow