K8s ingress with 2 domains both listening on port 80

3/18/2019

I am trying to replicate a Name based virtual hosting with two docker images in one deployment. Unfortunately I am only able to get 1 running due to a port conflict:

2019/03/19 20:37:52 [ERR] Error starting server: listen tcp :5678: bind: address already in use

Is it really not possible to have two images listening on the same port as part of the same deployment? Or am I going wrong elsewhere?

Minimal example adapted from here

# set up ingress
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml

# set up load balancer
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml

# spin up two containers in one deployment, same container port
kubectl apply -f test.yaml

test.yaml:

apiVersion: v1
kind: Service
metadata:
  name: echo1
spec:
  ports:
    - port: 80
      targetPort: 5678
  selector:
    app: echo1
---
apiVersion: v1
kind: Service
metadata:
  name: echo2
spec:
  ports:
    - port: 80
      targetPort: 5678
  selector:
    app: echo2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo12
spec:
  selector:
    matchLabels:
      app: echo12
  replicas: 1
  template:
    metadata:
      labels:
        app: echo12
    spec:
      containers:
        - name: echo1
          image: hashicorp/http-echo
          args:
            - "-text=echo1"
          ports:
            - containerPort: 5678
        - name: echo2
          image: hashicorp/http-echo
          args:
            - "-text=echo2"
          ports:
            - containerPort: 5678
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo-ingress
spec:
  rules:
    - host: echo1.example.com
      http:
        paths:
          - backend:
              serviceName: echo1
              servicePort: 80
    - host: echo2.example.com
      http:
        paths:
          - backend:
              serviceName: echo2
              servicePort: 80

Update: If I add a separate deployment it works. Is this by design or is there any way that I can achieve this in one deployment (reason: I'd like to be able to reset all deployed domains at once)?

-- Fooman
kubernetes
kubernetes-ingress

1 Answer

3/21/2019

Problem 1: Creating two different service backends in one pod of one deployment. This is not what the pods are designed for. If you want to expose multiple services, you should have a pod (at least) to back each service. Deployments wrap around the pod by allowing you to define replication and liveliness options. In your case, you should have one deployment (which creates one or multiple pods that will respond to one echo request) for its corresponding service.

Problem 2: You are not linking your services to your backends properly. The service clearly is trying to select a label app=echo or app=echo2. In your deployment, your app=echo12. Consequently, the service simply won't be able to find any active endpoints.

To address the above problems, try this below:

apiVersion: v1
kind: Service
metadata:
  name: echo1
spec:
  ports:
    - port: 80
      targetPort: 5678
  selector:
    app: echo1
---
apiVersion: v1
kind: Service
metadata:
  name: echo2
spec:
  ports:
    - port: 80
      targetPort: 5678
  selector:
    app: echo2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo1
spec:
  selector:
    matchLabels:
      app: echo1
  replicas: 1
  template:
    metadata:
      labels:
        app: echo1
    spec:
      containers:
        - name: echo1
          image: hashicorp/http-echo
          args:
            - "-text=echo1"
          ports:
            - containerPort: 5678
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo2
spec:
  selector:
    matchLabels:
      app: echo2
  replicas: 1
  template:
    metadata:
      labels:
        app: echo2
    spec:
      containers:
        - name: echo2
          image: hashicorp/http-echo
          args:
            - "-text=echo2"
          ports:
            - containerPort: 5678
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo-ingress
spec:
  rules:
    - host: echo1.example.com
      http:
        paths:
          - backend:
              serviceName: echo1
              servicePort: 80
    - host: echo2.example.com
      http:
        paths:
          - backend:
              serviceName: echo2
              servicePort: 80

I have tested the above in my own cluster and verified that it is working (with different ingress urls of course). Hope this helps!

-- Frank Yucheng Gu
Source: StackOverflow