So on GKE I have a Node.js app which for each pod uses about: CPU(cores): 5m, MEMORY: 100Mi
However I am only able to deploy 1 pod of it per node. I am using the GKE n1-standard-1 cluster which has 1 vCPU, 3.75 GB per node.
So in order to get 2 pods of app up total = CPU(cores): 10m, MEMORY: 200Mi, it requires another entire +1 node = 2 nodes = 2 vCPU, 7.5 GB to make it work. If I try to deploy those 2 pods on the same single node, I get insufficient CPU error.
I have a feeling I should actually be able to run a handful of pod replicas (like 3 replicas and more) on 1 node of f1-micro (1 vCPU, 0.6 GB) or f1-small (1 vCPU, 1.7 GB), and that I am way overprovisioned here, and wasting my money.
But I am not sure why I seem so restricted by insufficient CPU. Is there some config I need to change? Any guidance would be appreciated.
Allocatable:
cpu: 940m
ephemeral-storage: 47093746742
hugepages-2Mi: 0
memory: 2702216Ki
pods: 110
Non-terminated Pods: (7 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
default mission-worker-5cf6654687-fwmk4 100m (10%) 0 (0%) 0 (0%) 0 (0%)
default mission-worker-5cf6654687-lnwkt 100m (10%) 0 (0%) 0 (0%) 0 (0%)
kube-system fluentd-gcp-v3.1.1-5b6km 100m (10%) 1 (106%) 200Mi (7%) 500Mi (18%)
kube-system kube-dns-76dbb796c5-jgljr 260m (27%) 0 (0%) 110Mi (4%) 170Mi (6%)
kube-system kube-proxy-gke-test-cluster-pool-1-96c6d8b2-m15p 100m (10%) 0 (0%) 0 (0%) 0 (0%)
kube-system metadata-agent-nb4dp 40m (4%) 0 (0%) 50Mi (1%) 0 (0%)
kube-system prometheus-to-sd-gwlkv 1m (0%) 3m (0%) 20Mi (0%) 20Mi (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 701m (74%) 1003m (106%)
memory 380Mi (14%) 690Mi (26%)
Events: <none>After the deployment, check the node capacities with kubectl describe nodes. For e.g: In the code example at the bottom of the answer:
Allocatable cpu: 1800m
Already used by pods in kube-system namespace: 100m + 260m + +100m + 200m + 20m = 680m
Which means 1800m - 680m = 1120m is left for you to use
So, if your pod or pods request for more than 1120m cpu, they will not fit on this node
So in order to get 2 pods of app up total = CPU(cores): 10m, MEMORY: 200Mi, it requires another entire +1 node = 2 nodes = 2 vCPU, 7.5 GB to make it work. If I try to deploy those 2 pods on the same single node, I get insufficient CPU error.
If you do the exercise described above, you will find your answer. In case, there is enough cpu for your pods to use and still you are getting insufficient CPU error, check if you are setting the cpu request and limit params correctly. See here
If you do all the above and still it's an issue. Then, I think in your case, what could be happening is that you are allocating 5-10m cpu for your node app which is too less cpu to allocate. Try increasing that may be to 50m cpu.
I have a feeling I should actually be able to run a handful of pod replicas (like 3 replicas and more) on 1 node of f1-micro (1 vCPU, 0.6 GB) or f1-small (1 vCPU, 1.7 GB), and that I am way overprovisioned here, and wasting my money.
Again, do the exercise describe above to conclude that
Name: e2e-test-minion-group-4lw4
[ ... lines removed for clarity ...]
Capacity:
cpu: 2
memory: 7679792Ki
pods: 110
Allocatable:
cpu: 1800m
memory: 7474992Ki
pods: 110
[ ... lines removed for clarity ...]
Non-terminated Pods: (5 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
kube-system fluentd-gcp-v1.38-28bv1 100m (5%) 0 (0%) 200Mi (2%) 200Mi (2%)
kube-system kube-dns-3297075139-61lj3 260m (13%) 0 (0%) 100Mi (1%) 170Mi (2%)
kube-system kube-proxy-e2e-test-... 100m (5%) 0 (0%) 0 (0%) 0 (0%)
kube-system monitoring-influxdb-grafana-v4-z1m12 200m (10%) 200m (10%) 600Mi (8%) 600Mi (8%)
kube-system node-problem-detector-v0.1-fj7m3 20m (1%) 200m (10%) 20Mi (0%) 100Mi (1%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
680m (34%) 400m (20%) 920Mi (12%) 1070Mi (14%)