Connecting to Kubernetes TCP and UDP services on the same IP

12/15/2021

I have 2 pods in a Kubernetes namespace. One uses TCP and the other uses UDP and both are exposed using ClusterIP services via external IP. Both services use the same external IP.

This way I let my users access both the services using the same IP. I want to remove the use of spec.externalIPs but be able to allow my user to still use a single domain name/IP to access both the TCP and UDP services.

I do not want to use spec.externalIPs, so I believe clusterIP and NodePort services cannot be used. ​Load balancer service does not allow me to specify both TCP and UDP in the same service.

I have experimented with NGINX Ingress Controller. But even there the Load Balancer service needs to be created which cannot support both TCP and UDP in the same service.

Below is the cluster IP service exposing the apps currently using external IP:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: tcp-udp-svc
  name: tcp-udp-service
spec:
  externalIPs:
  - <public IP- IP2>
  ports:
  - name: tcp-exp
    port: 33001
    protocol: TCP
    targetPort: 33001
  - name: udp-exp
    port: 33001
    protocol: UDP
    targetPort: 33001
  selector:
    app: tcp-udp-app
  sessionAffinity: None
  type: ClusterIP

The service shows up like below

NAME                  TYPE           CLUSTER-IP            EXTERNAL-IP        PORT(S)
tcp-udp-service       ClusterIP      <internal IP IP1>     <public IP- IP2>   33001/TCP,33001/UDP

Using the above set up, both the TCP and UDP apps on port 33001 is accessible externally just fine using IP2.

As you can see I've used:

spec:
  externalIPs:
  - <public IP- IP2>

In the service to make it accessible externally.

However I do not want to use this set up, ie. I am looking for a set up without using the spec.externalIPs.

When using a load balancer service to expose the apps, I see that both TCP and UDP cannot be added in the same load balancer service. So I have to create one load balancer service for TCP and add another load balancer service for UDP like below

NAME                TYPE           CLUSTER-IP            EXTERNAL-IP        PORT(S)
tcp-service         LoadBalancer   <internal IP IP1>     <public IP- IP2>   33001/TCP

udp-service         LoadBalancer   <internal IP IP3>     <public IP- IP4>   33001/UDP


---
apiVersion: v1
kind: Service
metadata:
  name: tcp-service 
 spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: tcp-svc
    port: 33001
    protocol: TCP
    targetPort: 33001
  selector:
     app: tcp-udp-app
  sessionAffinity: None
  type: LoadBalancer

---
apiVersion: v1
kind: Service
metadata:
  name: udp-service
 spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: udp-svc
    port: 33001
    protocol: UDP
    targetPort: 33001
  selector:
     app: tcp-udp-app
  sessionAffinity: None
  type: LoadBalancer 

But the problem is that each of these services get individual IPs assigned (IP2 & IP4). But I want to be able to access both the TCP & UDP apps using the same IP. When testing out with nginx ingress controller too, I am faced the same issue as above.

Is there any other possible way to achieve what I am looking for, ie. to expose both TCP and UDP services on the same IP, but without using the spec.externalIPs?

-- anas
kubernetes
kubernetes-ingress

1 Answer

12/30/2021

Unfortunately, you will not be able to achieve your desired result with Load Balancer. Service type in any way for UDP traffic, because according the following documentation UDP protocol is not supported by any of VPC load balancer types.

You could theoretically define a portable public IP address for LoadBalancer Service type, by using the loadBalancerIP annotation, but this portable public IP address has to be available in portable public subnet upfront and Cloud Provivers's LB needs to support UDP protocol. You can see this doc

Workaround for non-Prod setup:

You can use hostPort to expose TCP & UDP ports directly on worker nodes. Could be used together with some Ingress controllers that support TCP & UDP Services, like NGINX Ingress. For more see this documentation.

-- Mikołaj Głodziak
Source: StackOverflow