K
Q

pod with app is getting evicted from gke as out of memory even if there is only 1/3rd memory used

9/2/2019

I am newbie to gke.

I have python app running inside a gke pod. Pod gets evicted as out of memory after 30minutes. Total vm memory is 13GB, and as i ssh into the pod, the peak used memory before eviction is only about 3GB...

I have tried running some dummy code as defined in Dockerfile "CMD tail -f /dev/null", then connect to the pod and running scraper script manually, with success - being able to finish with peak mem usage of 9 GB.

docker file:

CMD python3 scraper.py

> Managed pods Revision Name    Status  Restarts    Created on 1
>   scraper-df68b65bf-gbhms      Running    0   Sep 2, 2019, 2:59:59 PM 1
>   scraper-df68b65bf-gktqw      Running    0   Sep 2, 2019, 2:59:59 PM 1
>   scraper-df68b65bf-z4kjb      Running    0   Sep 2, 2019, 2:59:59 PM 1
>   scraper-df68b65bf-wk6td      Running    0   Sep 2, 2019, 3:00:45 PM 1
>   scraper-df68b65bf-xqm7h      Running    0   Sep 2, 2019, 3:00:45 PM

My guess is there are many instances of my app running inside of space of 13 GB in many parallel pods? How do I run single instance of my app on gke so I have all memory from vm available to it?

-- Wojtas.Zet
docker
google-cloud-platform
google-kubernetes-engine
kubernetes

Similar Questions

What does the devel flag do in kubectl?
How to create liveness and readiness probe for a java application - RADIUS
Helm Secrets Unable To Recognize Data
Standard way to monitor ingress traffic in K8 or EKS

2 Answers

9/2/2019

Do you have replica count set to one in your deployment.yaml file?

spec:
  replicas: 1

In case it is HorizontalPodAutoscaler you can edit it by:

  1. Get the HorizontalPodAutoscaler

    kubectl get HorizontalPodAutoscaler

  2. Edit it by using the edit command

    kubectl edit HorizontalPodAutoscaler <pod scaler name>

And the end result of HorizontalPodAutoscaler looks like this

spec:
  maxReplicas: 1
  minReplicas: 1
-- Bimal
Source: StackOverflow
9/2/2019

Awesome reply @Bismal.

@Wotjas, just to add my 2 cents; you can use the Cloud Console to set the min and max values, you just need to go to: Cloud Menu -> GKE -> Workloads -> Actions -> Scale

Set the desired values, then save. More detailed information can be found in this document [1].

[1] https://cloud.google.com/kubernetes-engine/docs/how-to/scaling-apps

-- JorgeHPM
Source: StackOverflow