Kubernetes Autoscaling with Memory Not Working, But working for CPU

11/21/2018

The yaml I used is shown below

    apiVersion: v1
    kind: Service
    metadata:
      name: xxx-svc
      labels:
        app: xxxxxx
    spec:
      type: NodePort
      ports:
      - port: 8080
      selector:
        app: xxxxxx
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: my-xxx
      labels:
        app: xxxxxx
    spec:
      selector:
        matchLabels:
          app: xxxxxx
      template:
        metadata:
          labels:
            app: xxxxxx
        spec:
          containers:
          - name: xxxxxx
            image: yyy/xxxxxx:latest
            ports:
            - containerPort: 8080
            resources:
              requests:
                cpu: "100m"
                memory: "504Mi"
              limits:
                cpu: "100m"
                memory: "504Mi"
    ---
    apiVersion: autoscaling/v2beta2
    kind: HorizontalPodAutoscaler
    metadata:
      name: xxxxxx
    spec:
      scaleTargetRef:
        apiVersion: apps/v1
        kind: Deployment
        name: my-xxx
      minReplicas: 1
      maxReplicas: 3
      metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Resource
    resource:
      name: memory
      target:
        type: Value
        averageValue: 500Mi

Service, HPA, Deployment everything deployed successfully, but when I check hpa (kubectl get hpa) I am getting below result

NAME        REFERENCE              TARGETS                   MINPODS   
MAXPODS   REPLICAS   AGE

xxxxxx   Deployment/my-xxx   unknown/500Mi, 1%/50%   1         3         3          69m

The reason I got (kubectl describe hpa) is

Warning FailedComputeMetricsReplicas 21m (x4 over 22m) horizontal-pod-autoscaler failed to get memory utilization: missing request for memory

What might be the reason that memory is Unknown but CPU is working

-- JibinNajeeb
docker
kubernetes

2 Answers

8/9/2019

Kubernetes auto scaling for memory is not applied yet. You can write a script which tops on the desired pods and get the memory value and divide it to the desired memory to get percentage and scale up or scale down pods. Just run the script in a cron job which keeps on checking the stats in a frequent interval.

-- rajdeepbs29
Source: StackOverflow

11/21/2018

The reason for this:

Warning FailedComputeMetricsReplicas 21m (x4 over 22m) horizontal-pod-autoscaler failed to get memory utilization: missing request for memory

Kubernetes HPA does not work by default with memory you need to create custom metric for memory and then use it. I found some additional information here how people try to solve same issue.

Pod Memory Based AutoScaling

In this section, we are discussing how you can deploy autoscaling on the basis of memory that pods are consuming. We have used the command “kubectl top pod” to get the utilized pod memory and applied the logic.

  • Get the average pod memory of the running pods: Execute the script as follows:
-- Nick Rak
Source: StackOverflow