K
Q

In Kubernetes, how to make sure one pod only serve one connection

2/24/2020

Situation

I want to have a pool of serving pods (say I have 50 serving pods lying around). They will be exposed via a LoadBalancer Service.

I want to make sure that:

  1. each pod serves only one TCP connection and keep this connection alive until the client terminates it. Until the end of this pod's life, it will not receive any other TCP connections.
  2. Once the client terminates the connection, the pod cleans up and destroys it self.
  3. Another pod is spinned up to match the desired replica number. Since it's not currently serving any TCP connection, it can be chosen to serve the next TCP connection that goes into the LoadBalancer service.

Example

1. Initially, a `Deployment` specifies a pool of 2 pods behind a LoadBalancer Service.

[1] [2]


------LoadBalancer-------



2. A client initiates a TCP connection to the LoadBalancer (e.g. telnet loadbalancer.domain.com 80) and the TCP connection is routed to the first vacant pod.

[1] [2]
 |
 |
------LoadBalancer-------
 |
 |
cl1



3. 24 hours after that (assuming data has been passing between client1 & pod1), another client hits to the same Load Balancer public domain. Since pod1 is serving client1, I want the second client to be routed to another vacant pod, such as pod 2.

[1] [2]
 |   |
 |   |
------LoadBalancer-------
 |   |
 |   |
cl1 cl2



4. 24 hours after that, client 1 terminates the connection, I want pod1 to do clean up and destroy itself shortly afterwards. No new connections should be routed to it. That leaves pod2 the only one still running.

[2]
 |
 |
------LoadBalancer-------
 |
 |
cl2



5. `Deployment` will create additional pods to ensure the number of replicas. So pod3 is created.

[1] [3]
 |   
 |   
------LoadBalancer-------
 |   
 |   
cl1 



6. Another client hits the same endpoint and is routed to a vacant pod (pod 3 in this case).

[1] [3]
 |   |
 |   |
------LoadBalancer-------
 |   |
 |   |
cl1 cl3

And so on and so forth.

Anybody has any ideas how this can be solved on K8s?

-- Tran Triet
kubernetes

Similar Questions

Unable to create wildcard (*) Cert with Kubernetes and Letsencrypt using Azure DNS zone
Pulling docker image from private github package using AWS EKS with AWS Fargate
Expose K8s POD to single/range of External IPs
How to debug crash-looping pods in OpenShift?

4 Answers

2/25/2020

You could deploy the pods with StatefulSet so you could use Headless Service.

A Headless Service does not contain a ClusterIP. Instead, it creates several Endpoints that are used to produce DNS records. Each DNS record is bound to a pod. All of this is done internally by Kubernetes, but it’s good to have an idea about how it does it.

You can read more about Kubernetes StatefulSets 101 - State of the Pods.

You then can implement a script (deployed in front of the StatefulSet) that will route client to one DNS record.

You could also have a look for some message queues brokers.

There are KubeMQ, RabbitMQ and Redis. There is also Pub/Sub on GCP and AWS.

-- Crou
Source: StackOverflow
2/24/2020

I think it's possible to achieve this with a serverless technology on top of kubernetes. Check Knative serving which is a serverless technology. But in this case there is no pool of pods pre created, pods are created on demand when client request comes and gets destroyed after the request finishes.

-- Arghya Sadhu
Source: StackOverflow
2/24/2020

So I guess this would work.

First set a ReadinessProbe to poll an endpoint/tcp port on your pod very agressivly (as often as possible).

Then as soon as you get a connection make sure that the ReadinessProbe fails, but at the same time make sure the LivenessProbe DOESN'T fail.

Finally, after the client disconnects terminate the application.

Your deplopment needs to have enough replicas to serve all clients that can enter simultanious since one pod will never serve two clients.

-- Andreas Wederbrand
Source: StackOverflow
2/24/2020

Have you looked at k8s' sessionAffinity, it might help I think. This link says:

kube-proxy takes the SessionAffinity setting of the Service into account when deciding which backend Pod to use.

and

If you want to make sure that connections from a particular client are passed to the same Pod each time, you can select the session affinity based on the client’s IP addresses by setting service.spec.sessionAffinity to “ClientIP” (the default is “None”). You can also set the maximum session sticky time by setting service.spec.sessionAffinityConfig.clientIP.timeoutSeconds appropriately.

After the client terminates, make the server return a failure code at a path like /clientstatus, use readinessProbe/livenessProbe to check this path every second, and delete the pod if readinessProbe/livenessProbe fails.

-- Vikram Hosakote
Source: StackOverflow