ingress points to wrong port on a service

10/5/2018

I have a Kubernetes service that exposes two ports as follows

Name:              m-svc
Namespace:         m-ns
Labels:            
Annotations:       <none>
Selector:          app=my-application
Type:              ClusterIP
IP:                10.233.43.40
Port:              first  8080/TCP
TargetPort:        8080/TCP
Endpoints:         10.233.115.178:8080,10.233.122.166:8080
Port:              second  8888/TCP
TargetPort:        8888/TCP
Endpoints:         10.233.115.178:8888,10.233.122.166:8888
Session Affinity:  None
Events:            <none>

And here is the ingress definition:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: f5
    virtual-server.f5.com/http-port: "80"
    virtual-server.f5.com/ip: controller-default
    virtual-server.f5.com/round-robin: round-robin
  creationTimestamp: 2018-10-05T18:54:45Z
  generation: 2
  name: m-ingress
  namespace: m-ns
  resourceVersion: "39557812"
  selfLink: /apis/extensions/v1beta1/namespaces/m-ns
  uid: 20241db9-c8d0-11e8-9fac-0050568d4d4a
spec:

  rules:
  - host: www.myhost.com
    http:
      paths:
      - backend:
          serviceName: m-svc
          servicePort: 8080
        path: /first/path
      - backend:
          serviceName: m-svc
          servicePort: 8080
        path: /second/path
status:
  loadBalancer:
    ingress:
    - ip: 172.31.74.89

But when I go to www.myhost.com/first/path I end up at the service that is listening on port 8888 of m-svc. What might be going on?

Another piece of information is that I am sharing a service between two ingresses that point to different ports on the same service, is this a problem? There is a different ingress port the port 8888 on this service which works fine

Also I am using an F5 controller

After a lot of time investigating this, it looks like the root cause is in the F5s, it looks like because the name of the backend (Kubernetes service) is the same, it only creates one entry in the pool and routes the requests to this backend and the one port that gets registered in the F5 policy. Is there a fix for this? A workaround is to create a unique service for each port but I dont want to make this change , is this possible at the F5 level?

-- user_mda
kubernetes
kubernetes-ingress
kubernetes-service

2 Answers

10/6/2018

Your Service appears to be what is called a headless service: https://kubernetes.io/docs/concepts/services-networking/service/#headless-services. This would explain why the Endpoints was created automatically.

Something is amiss because it should be impossible for your HTTP request to arrive at you pods without the .spec.selector populated.

I suggest deleting the Service you created and delete the Endpoints with the same name and then recreate the Service with type=ClusterIP and the spec.selector properly populated.

-- mpalumbo7
Source: StackOverflow

10/5/2018

From what I see you don't have a Selector field in your service. Without it, it will not forward to any backend or pod. What makes you think that it's going to port 8888? What's strange is that you have Endpoints in your service. Did you manually create them?

The service would have to be something like this:

Name:              m-svc
Namespace:         m-ns
Labels:            
Annotations:       <none>
Selector:          app=my-application
Type:              ClusterIP
IP:                10.233.43.40
Port:              first  8080/TCP
TargetPort:        8080/TCP
Endpoints:         10.233.115.178:8080,10.233.122.166:8080
Port:              second  8888/TCP
TargetPort:        8888/TCP
Endpoints:         10.233.115.178:8888,10.233.122.166:8888
Session Affinity:  None
Events:            <none>

Then in your deployment definition:

selector:
  matchLabels:
    app: my-application

Or in a pod:

apiVersion: v1
kind: Pod
metadata:
  annotations: { ... }
  labels:                                
    app: my-application

You should also be able to describe your Endpoints:

$ kubectl describe endpoints m-svc
Name:         m-svc
Namespace:    default
Labels:       app=my-application
Annotations:  <none>
Subsets:
  Addresses:          x.x.x.x
  NotReadyAddresses:  <none>
  Ports:
    Name    Port  Protocol
    ----    ----  --------
    first   8080  TCP
    second  8081  TCP

Events:  <none>
-- Rico
Source: StackOverflow