K
Q

How to do Dynamic Proxying to Kubernetes Pods?

2/10/2020

I'm wanting to create a service that can do some kind of dynamic proxying back to Kubernetes Pods. Basically I'll have hundreds of K8s Pods that are running the same application that map to a random port on the host (like 10456). However, each Pod is unique and I want traffic directed at a specific pod based on hostname. So when a request comes in for abc123.app.com, I'll have a proxy layer that does a lookup in a database to find what host and port that domain is running on (like 10.0.0.5:10456), then forward the request there. Is there a service that supports this? I've worked with Nginx a lot before, but I'm not clear if it could support this lookup functionality.

Has anyone built something like this before? what's the best way to build a proxy layer that can do lookups like that? How would I update the database when a pod moves from one host to another?

Thanks in advance!

EDIT:

I should have put this in there the first time, but the types of traffic going to these pods are RPC traffic and Peer to Peer traffic

-- gkrizek
docker
http
kubernetes
nginx
proxy

Similar Questions

Issues in Deployment.yaml file
Multiple commands in deployment yaml of Kubernetes
How to configure nginx deployment to pass traffic to front end deployment in Google Kubernetes Engine?
What can I place inside docker containers?

1 Answer

2/10/2020

You're describing something very similar to what kubernetes ingress definitions do for http traffic.

An ingress definition configures an ingress controller to point requests for a hostname at a service. The service selects endpoints (pods) via label selectors. When pods move, kubernetes updates the service automatically.

The work on your end just becomes pushing out config changes from your database via one of the API clients to kubernetes rather than directing a proxy. If your environment was extremely dynamic requiring reconfiguration all the time or you need to make dynamic decisions about where traffic should go, you might want to continue looking at a custom proxy or istio, openresty.

It sounds like you have unique deployments going to kubernetes already, so in addition to that include a service and ingress definition.

A simple example including a label on the a pod, a service that use the label. Then an ingress definition using the service.

apiVersion: v1
kind: Pod
metadata:
  name: my-app
  labels:
    app: host-abc123
spec:
  containers:
  - name: host-abc123
    image: me/my-app:1.2.1
    ports:
    - containerPort: 10456
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: host-abc123
spec:
  rules:
  - host: abc123.bar.com
    http:
      paths:
      - backend:
          serviceName: host-abc123
          servicePort: 80
apiVersion: v1
kind: Service
metadata:
  name: host-abc123
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 10456

The single ingress definition could include all hosts but I'm not sure how kubernetes and the ingress controllers would go replacing that regularly.

There are nginx based ingress controllers too. You end up with a nginx server config per ingress/host definition.

-- Matt
Source: StackOverflow