K
Q

Monitor percentage of CPU and memory in GKE nodes

1/8/2020

I would like to use Stackdriver monitoring to watch for my GKE nodes' CPU/mem % usage.

However, when going through the Stackdriver Monitoring UI, there does not seem to be any such option available (meaning in terms of percentage).

On the other hand there seem to be for example 2 extremely similar metrics (their description is identical) for (what appears to be) absolute mem usage. (screenshots attached).

Is anyone aware how one can provide for percentage of memory and CPU used on GKE nodes? (not containers)

enter image description here

enter image description here

-- pkaramol
google-cloud-monitoring
google-cloud-stackdriver
google-kubernetes-engine
monitoring
stackdriver

Similar Questions

Istio Egress MTLs for external services
Expose kibana (eck operator) via istio gateway - https problem (Error 503)
Kubernetes Let's Encrypt Certificate Not Issuing
Is it a good practice (feasible) to combine both nodeAfinity and podAffinity rules when assigning Pods to Nodes on kubernetes

2 Answers

1/8/2020

Unfortunately, it's not possible. Please have a look at the documentation page where you can find list of metrics that can be collected from Google Kubernetes Engine and then used with Stackdriver Monitoring. In addition you can also check these document to find some extra information about available metrics.

Meanwhile, you can find utilization at Kubernetes Engine -> Clusters -> Cluster -> Nodes -> Node details or just use a command line:

kubectl top nodes                                            
NAME                                       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
gke-test-cluster-default-pool-cd22e088-29sd   45m          4%     647Mi           24%       
gke-test-cluster-default-pool-cd22e088-8xsg   93m          9%     693Mi           26%       
gke-test-cluster-default-pool-cd22e088-t1h0   41m          4%     625Mi           23%

Also, you can solve this problem by using other 3rd party monitoring solutions.

-- Serhii Rohoza
Source: StackOverflow
1/8/2020

per https://cloud.google.com/monitoring/api/metrics_kubernetes :

node/memory/allocatable_utilization and node/cpu/allocatable_utilization seem to be doing what you want:

  • node/memory/allocatable_utilization - [BETA] Memory allocatable utilization

    • Description: The fraction of the allocatable memory that is currently in use on the instance. This value cannot exceed 1 as usage cannot exceed allocatable memory bytes. Sampled every 60 seconds. After sampling, data is not visible for up to 120 seconds.
    • Kind: GAUGE, Type: DOUBLE, Unit: 1
    • Monitored resource: k8s_node
    • Labels:
      • memory_type: Either evictable or non-evictable. Evictable memory is memory that can be easily reclaimed by the kernel, while non-evictable memory cannot.
      • component: Name of the respective system daemon.

  • node/cpu/allocatable_utilization - [BETA] CPU allocatable utilization.

    • Description: The fraction of the allocatable CPU that is currently in use on the instance. Sampled every 60 seconds. After sampling, data is not visible for up to 240 seconds.
    • Kind: GAUGE, Type: DOUBLE, Unit: 1
    • Monitored resource: k8s_node

Both metrics are marked as BETA, though.

original answer mentioned: container/cpu/limit_utilization and container/memory/limit_utilization

-- c69
Source: StackOverflow