I am trying to run horizontalpodautoscaler resource defining mix and max replica as 1. somehow once it started to run 2 instances of the pod are running in the same time then 1 pod is terminated. End state I have 1 pod running.
Is this the normal behaviour of HPA resource that somehow its created replicaset with 2 pods even the maximum is one
Thanks
Generally it will not happen. The only reason it could occur if hpa target metric target is very low and starting the application is crossing the target CPU threshold causing scaling of pods.
You can check the HPA status, events section
can explain the reason for scaling.
HPA status: kubectl describe HPA_NAME
Check below example for reference
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): 40% (406m) / 50%
Min replicas: 10
Max replicas: 100
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale the last scale time was sufficiently old as to warrant a new scale
ScalingActive True ValidMetricFound the HPA was able to succesfully calculate a replica count from cpu resource utilization (percentage of request)
ScalingLimited False DesiredWithinRange the desired replica count is within the acceptible range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulRescale 35m (x1216 over 12d) horizontal-pod-autoscaler New size: 10; reason: All metrics below target
These are the events flow from all relevant resources:
HPA Events:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
example-ms Deployment/example-ms <unknown>/50% 1 1 0 0s
example-ms Deployment/example-ms <unknown>/50% 1 1 0 1s
example-ms Deployment/example-ms <unknown>/50% 1 1 2 3s
example-ms Deployment/example-ms <unknown>/50% 1 1 1 33s
Replica Set Events:
NAME DESIRED CURRENT READY AGE
example-ms-59c9b45565 1 1 0 1s
example-ms-76896c7f7f 1 1 1 2s
example-ms-59c9b45565 1 1 1 2s
example-ms-76896c7f7f 1 1 1 32s
example-ms-59c9b45565 1 1 1 32s
example-ms-76896c7f7f 0 1 1 32s
example-ms-76896c7f7f 0 1 1 32s
example-ms-76896c7f7f 0 0 0 32s
Pod Events:
NAME READY STATUS RESTARTS AGE
example-ms-59c9b45565-6b5v8 0/1 ContainerCreating 0 0s
example-ms-76896c7f7f-cth7x 1/1 Running 0 2s
example-ms-59c9b45565-6b5v8 1/1 Running 0 2s
example-ms-59c9b45565-d2k8w 0/1 Terminating 0 2m
example-ms-59c9b45565-d2k8w 0/1 Terminating 0 2m
example-ms-59c9b45565-d2k8w 0/1 Terminating 0 2m
example-ms-59c9b45565-d2k8w 0/1 Terminating 0 2m
example-ms-76896c7f7f-cth7x 1/1 Terminating 0 32s
example-ms-76896c7f7f-cth7x 0/1 Terminating 0 1m
example-ms-76896c7f7f-cth7x 0/1 Terminating 0 1m
example-ms-76896c7f7f-cth7x 0/1 Terminating 0 1m
Deployment Spec:
spec:
minReadySeconds: 30
progressDeadlineSeconds: 120
replicas: 1
revisionHistoryLimit: 15
selector:
Deployment status:
tatus:
availableReplicas: 1
conditions:
- lastTransitionTime: 2019-04-21T08:51:03Z
lastUpdateTime: 2019-04-21T08:51:35Z
message: ReplicaSet "example-ms-59c9b45565" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: 2019-04-21T16:53:02Z
lastUpdateTime: 2019-04-21T16:53:02Z
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 3
readyReplicas: 1
replicas: 1
updatedReplicas: 1