Kubernetes auto-scaling nodes over AWS
I am working in set up a kubernetes cluster using the following stuff:
- AWS as a cloud provider
- kops (Version 1.6.0-alpha, just to test) as a cli tool to create and manage cluster
- kubectl (server : v1.6.2 and client : 1.6.0 ) to control my cluster
- Ubuntu 16 as a local OS
I have a simple k8s cluster with the following stuff:
- AWS region : us-west-2
- One master over : t2.medium / k8s-1.5-debian-jessie-amd64-hvm-ebs-2017-01-09
- One node onver : t2.medium / k8s-1.5-debian-jessie-amd64-hvm-ebs-2017-01-09
I also have some pods deployed over the cluster and I created jmeter stress test to generate artificial traffic.
My question is How can I create a auto-scaling node on a k8s cluster using kops over aws?
I just found the following ad-don kops addons in kops repository. I deployed as the docs says and it is available.
My parameters were:
CLOUD_PROVIDER=aws
IMAGE=gcr.io/google_containers/cluster-autoscaler:v0.4.0
MIN_NODES=1
MAX_NODES=3
AWS_REGION=us-east-2
GROUP_NAME="<the-auto-scaling-group-Name>"
SSL_CERT_PATH="/etc/ssl/certs/ca-certificates.crt" # (/etc/ssl/certs for gce)
$ kubectl get deployments --namespace=kube-system
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
cluster-autoscaler 1 1 1 1 3h
dns-controller 1 1 1 1 3h
kube-dns 2 2 2 2 3h
kube-dns-autoscaler 1 1 1 1 3h
kubernetes-dashboard 1 1 1 1 3hHowever, after stress my node using a pod with stress containers nothing happens (100% cpu utilization) and my auto-scaling group is not modified.
In the other hand, I export the kops output in terraform but there ia not auto scaling policies to generate auto-scaling base on cpu utilization.
Finally, I could find an entry in the k8s blog which indicates that it will be support in the future by AWS but there is not other announcement about it.
Any suggestion or experience with that task in AWS and kops?. Next I will try to generate auto-scaling policies manually just to test.
Similar Questions
2 Answers
After review kops (open issues related with auto scaling) I could not found an option for nodes auto scaling and as I wrote in my question I was looking for node auto scaling. Maybe in new versions of kops it will be consider. However, I decided set up a kubernetes v1.5.4 from scratch using terraform contemplating auto scaling in nodes. If some is interested in my implementation the source code is in my personal repo :: kubernetes cluster v1 with terraform (afym).
I would use this base to setup the cluster in production. I hope it can help someone.
Thank you and if someone find the auto scaling configuration option in kops it will be great.
Firstly you should use autoscaler gcr.io/google_containers/cluster-autoscaler:v0.5.0 when using Kubernetes 1.6.x .
Secondly from my understanding the autoscaler itself only scales the ASG if there is a pod in Pending state because it can't fit in any existing node.
For your use-case, Horizontal Pod Autoscaling will scale up your application (which is being stressed) when under high load, make sure to mention the requests portion in the podspec. Once the autoscaler sees newly scaled pods don't fit a node, it will launch new node.
Disclaimer: I haven't played with Horizontal Pod Autoscaling yet.