Unable to access Kubernetes ClusterIP services via nginx-ingress-controller

3/15/2019

I'm a Kubernetes amateur trying to use NGINX ingress controller on GKE. I'm following this google cloud documentation to setup NGINX Ingress for my services, but, I'm facing issues in accessing the NGINX locations.

What's working?

  1. Ingress-Controller deployment using Helm (RBAC enabled)
  2. ClusterIP service deployments

What's not working?

  1. Ingress resource to expose multiple ClusterIP services using unique paths (fanout routing)

K8S Services

[msekar@ebs kube-base]$ kubectl get services -n payment-gateway-7682352
NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
nginx-ingress-controller        LoadBalancer   10.35.241.255   35.188.161.171   80:31918/TCP,443:31360/TCP   6h
nginx-ingress-default-backend   ClusterIP      10.35.251.5     <none>           80/TCP                       6h
payment-gateway-dev             ClusterIP      10.35.254.167   <none>           5000/TCP                     6h
payment-gateway-qa              ClusterIP      10.35.253.94    <none>           5000/TCP                     6h

K8S Ingress

[msekar@ebs kube-base]$ kubectl get ing -n payment-gateway-7682352
NAME                HOSTS     ADDRESS          PORTS     AGE
pgw-nginx-ingress   *         104.198.78.169   80        6h

[msekar@ebs kube-base]$ kubectl describe ing pgw-nginx-ingress -n payment-gateway-7682352
Name:             pgw-nginx-ingress
Namespace:        payment-gateway-7682352
Address:          104.198.78.169
Default backend:  default-http-backend:80 (10.32.1.4:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
        /dev/   payment-gateway-dev:5000 (<none>)
        /qa/    payment-gateway-qa:5000 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"pgw-nginx-ingress","namespace":"payment-gateway-7682352"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"payment-gateway-dev","servicePort":5000},"path":"/dev/"},{"backend":{"serviceName":"payment-gateway-qa","servicePort":5000},"path":"/qa/"}]}}]}}

  kubernetes.io/ingress.class:               nginx
  nginx.ingress.kubernetes.io/ssl-redirect:  false
Events:                                      <none>

Last applied configuration in the annotations (ingress description output) shows the ingress resource manifest. But, I'm pasting it below for reference

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: pgw-nginx-ingress
  namespace: payment-gateway-7682352
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: payment-gateway-dev
          servicePort: 5000
        path: /dev/
      - backend:
          serviceName: payment-gateway-qa
          servicePort: 5000
        path: /qa/

Additional Info

The services I'm trying to access are springboot services that use contexts, so, the root location isn't a valid end-point.

The container's readiness and liveliness probes are defined accordingly.

For example, "payment-gateway-dev" service is using a context /pgw/v1 context, so, the deployment can only be accessed through the context. To access application's swagger spec you would use the URL

http://<>/pgw/v1/swagger-ui.html

Behaviour of my deployment

ingress-controller-LB-ip = 35.188.161.171

Any suggestions about or insights into what I might be doing wrong will be much appreciated.

-- manikandan sekar
kubernetes
kubernetes-ingress
nginx

2 Answers

1/4/2020

An alternate approach would be to use host based routing. You can simply create a couple CNAME DNS entries for the Ingress static IP (pgw-dev.foo.com, pgw-qa.foo.com) and replace your path: attributes with host: attributes. No URL rewrites necessary.

The best reason for using the host based approach, imo, is clarity and flexibility for humans. I've worked in a lot of different places. Almost all of them have used host names to differentiate environments in this way. Tastes great, less filling.

For example, if you split DEV and QA onto separate clusters, no one has to change their configs (and your K8s templates will be reusable). Just update DNS. If you want to spin up a new Staging or Performance Test environment, again, your existing test harnesses should be very easily adapted to the new environment: just change the host name in the config.

Over time, I think you'll find hostname is a more natural way to distinguish environments than a path prefix.

-- Charlie Reitzel
Source: StackOverflow

3/16/2019

+1 for this well asked question.

Your setup seemed right to me. In you explanation, I could find that your services would require http://<>/pgw/v1/swagger-ui.html as context. However, in your setup the path submitted to the service will be http://<>/qa/pgw/v1/swagger-ui.html if your route is /qa/.

To remove the prefix, what you would need to do is to add a rewrite rule to your ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: pgw-nginx-ingress
  namespace: payment-gateway-7682352
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: payment-gateway-dev
          servicePort: 5000
        path: /dev/(.+)
      - backend:
          serviceName: payment-gateway-qa
          servicePort: 5000
        path: /qa/(.+)

After this, you service should receive the correct contexts.

Ref:

-- Fei
Source: StackOverflow