I've got a k8s deployment with 3 pods in it and I've set up a NodePort service to forward SSH (port 22) to the 3 pods. Everything works as expected, but each time I SSH in, I get a random pod. I'd like to make it sticky so that I always get the same pod, but I'm unsure if this is possible.
According to the documentation, setting sessionAffinity: ClientIP probably won't work for NodePorts. I don't think externalTrafficPolicy: Local will work because you need to use a LoadBalancer service. I don't think LoadBalancer services are feasible for me because I need to create hundreds of these and each LoadBalancer costs money and uses up quota.
What I'm wondering here is whether it's possible to create a service that doesn't point to all 3 pods in the deployment, but instead exactly 1 pod. That would help for my situation. I could manually attach a special label to 1 pod and set the service selector to that label, but it feels brittle to me in case that pod dies and is replaced.
One way to get around this would be to create your pods using StatefulSet instead of deployment. Then your pods have a deterministic names, and when restarted, will retain their name. That way you can create a service that points to myapp-0, myapp-1 etc. and be reasonably sure that interruptions, while will break for a while when pod is rescheduled/restarted, will also get back to a working state. You will need to handle automation of such service creation when scaling StatefulSet though and your "affinity" would be based on service port that client is connecting to (can't have multiple services on same port)
That said, this is definitely not a good pattern to follow. You should ensure that your client can connect to any of the pods and that they share required state by means of another service they all use or a shared RWX volume if it's about files.