Kubernetes ingress with 2 services does not always find the correct service

10/9/2018

I have a Kubernetes cluster with a backend service and a security service. The ingress is defined as follows:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: solidary-life
  annotations:
    kubernetes.io/ingress.global-static-ip-name: sl-ip
    certmanager.k8s.io/acme-http01-edit-in-place: "true"
    ingress.kubernetes.io/force-ssl-redirect: "true"
    ingress.kubernetes.io/ssl-redirect: "true"
  labels:
    app: sl
spec:
  rules:
    - host: app-solidair-vlaanderen.com
      http:
        paths:
        - path: /v0.0.1/*
          backend:
            serviceName: backend-backend
            servicePort: 8080
        - path: /auth/*
          backend:
            serviceName: security-backend
            servicePort: 8080
  tls:
  - secretName: solidary-life-tls
    hosts:
    - app-solidair-vlaanderen.com

The backend service is configured like:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: backend
  labels:
    app: sl
spec:
  template:
    metadata:
      labels:
        app: sl
        tier: web
    spec:
      containers:
      - name: backend-app
        image: gcr.io/solidary-life-218713/sv-backend:0.0.6
        ports:
          - name: http
            containerPort: 8080
        readinessProbe:
          httpGet:
            path: /v0.0.1/api/online
            port: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: backend-backend
  labels:
    app: sl
spec:
  type: NodePort
  selector:
    app: sl
    tier: web
  ports:
  - port: 8080
    targetPort: 8080

and the auth server service:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: security
  labels:
    app: sl-security
spec:
  template:
    metadata:
      labels:
        app: sl
        tier: web
    spec:
      containers:
      - name: security-app
        image: gcr.io/solidary-life-218713/sv-security:0.0.1
        ports:
          - name: http
            containerPort: 8080
          - name: management
            containerPort: 9090
          - name: jgroups-tcp
            containerPort: 7600
          - name: jgroups-tcp-fd
            containerPort: 57600
          - name: jgroups-udp
            containerPort: 55200
            protocol: UDP
          - name: jgroups-udp-mc
            containerPort: 45688
            protocol: UDP
          - name: jgroups-udp-fd
            containerPort: 54200
            protocol: UDP
          - name: modcluster
            containerPort: 23364
          - name: modcluster-udp
            containerPort: 23365
            protocol: UDP
          - name: txn-recovery-ev
            containerPort: 4712
          - name: txn-status-mgr
            containerPort: 4713
        readinessProbe:
          httpGet:
            path: /auth/
            port: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: security-backend
  labels:
    app: sl
spec:
  type: NodePort
  selector:
    app: sl
    tier: web
  ports:
  - port: 8080
    targetPort: 8080

Now I can go to the url's:

Sometimes this works, sometimes I get 404's. This is quite annoying and I am quite new to Kubernetes. I don't find the error.

Can it have something to do with the "sl" label that's on both the backend and security service definition?

-- Vandeperre Maarten
google-kubernetes-engine
kubernetes
kubernetes-ingress

1 Answer

10/9/2018

Yes. At least that must be the start of the issue, assuming all your services are on the same Kubernetes namespace. Can you use a different label for each?

So, in essence, you have 2 services that are randomly selecting pods belonging to the security Deployment and the backend deployment. One way to determine where your service is really sending requests is by looking at its endpoints and running:

kubectl -n <your-namespace> <get or describe> ep 
-- Rico
Source: StackOverflow