I have an ingress resource like the following:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: "app-ingress"
labels:
app: "app"
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
cert-manager.io/cluster-issuer: letsencrypt
spec:
tls:
- hosts:
- api.example.com
secretName: tls-secret
rules:
- host: api.example.com
http:
paths:
- path: /grpc
pathType: ImplementationSpecific
backend:
service:
name: "app-server"
port:
number: 50051
- path: /callback
pathType: ImplementationSpecific
backend:
service:
name: "app-server"
port:
number: 80
app-server
runs two services (on the same pod), a grpc
web server and a http
web server. To start, a client makes a request to api.example.com/grpc
. To fulfill the request, the server makes a request to a third party service, waits for a callback on api.example.com/callback
, and then completes the client's request (api.example.com/callback
is only accessed by the third party service, not the client).
The question is, how can I specifically route api.example.com/callback
back to the same pod that made the initial request? Obviously if pod1
were to make a request to the third party service and pod2
were to receive the callback, pod1
would never receive the callback and the client request would have to timeout. I can attach information to the callback I receive from the third party service, so is there a way I could attach some sort of pod id so that kubernetes will route the callback back to the pod that initiated it?
I've looked at maybe using StatefulSets
, but I'm not sure exactly how this would work. Or would it be better to have sort of "notification pod" that receives callbacks and routes them to the correct pod?
One way would be to user sticky sessons.
If you want to make sure that connections from a particular client are passed to the same Pod each time, you can select the session affinity based on the client's IP addresses by setting service.spec.sessionAffinity
to "ClientIP" (the default is "None"). You can also set the maximum session sticky time by setting service.spec.sessionAffinityConfig.clientIP.timeoutSeconds
appropriately (the default value is 10800, which works out to be 3 hours).
Another way is to design the Application in away that it doesn't matter which pod receives callback request, the proper pod with the pending client connection will be notified (and provided with data), and send a response back to the client. To communicate requests data you can use some sort of messaging service or distributed in-memory data stores like Redis.