I have created an autoscaling Kubernetes cluster on Google Cloud Platform. I have a use case where I want to launch dedicated pods on each node .i.e. each worker node can have only one such pod and I want these dedicated pods to be launched on the newly created nodes which were formed due to autoscaling. Is there a way I can achieve this. for e.g. If I have 3 workers nodes , I will specify number of replicas to be 3 along with podAntiAffinity in my deployment file so that each of these pods will launch on 3 different nodes. But If my cluster autoscales and a 4th node is added, how can I ensure that this Pod will be added on the 4th node?
If you need one pod on every node, a DaemonSet is the tool you're looking for. If you add nodes, the DaemonSet controller will launch new copies of the pod there automatically.
This is a good approach for tools like monitoring services and service meshes, where the host itself is an important entity and you really do want one per host (to collect per-host disk I/O, to transparently encrypt inter-host network traffic). You should stick to an ordinary Deployment for more typical server-based workloads where the number of replicas should scale independently from the number of hosts (you could have 20 hosts and 4 pod replicas; or you could schedule 20 pod replicas on 4 8-core systems).