I'm trying to setup k8s autoscaler, I went through: https://hub.helm.sh/charts/stable/cluster-autoscaler and did a lot of experiment but cannot make it work. I have two choices but both not working:
helm install my-release stable/cluster-autoscaler --set autoDiscovery.clusterName=hide-cluster,rbac.serviceAccount.name=default,tolerations[0].effect=NoSchedule,tolerations[0].operator="Equal",tolerations[0].value=/"true/",tolerations[0].key=node-role.kubernetes.io/master
but it always says the field tolerations[0].value=/"true/"
is not as expected, I tried several possible versions I can think of, like using 'true', "true", true etc. none of them works.There is very very limited document about how to setup autoscaler without using kops or EKS. I also wonder how many autoscaler solution we have on the market? It seems not many learning resources online for autoscaler.
Edit: I finally make the autoscaler run following this guide: https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html
But it does not include EC2 instances created by ASG to nodes. I checked the log it says the ASG is already found and registered, no error or other hint I can find why no nodes shows up in kubectl get nodes
It looks like auto-scaler does not work with standalone k8s but have to use on top of EKS or Kops, although I'm not sure. really too limited resource available.
Try to attach these instances to ASG - follow this tutorial: attaching-instances-asg.
Also you have to remember that instance to attach must meet the specific criteria:
State of the instance is running
The AMI used to launch the instance still exist.
The instance is not a member of different ASG
The instance is launched into one of the Availability Zones defined in your ASG
If the Auto Scaling group has an attached load balancer, the instance and the load balancer must both be in EC2-Classic or the same VPC. If the Auto Scaling group has an attached target group, the instance and the load balancer must both be in the same VPC.