K
Q

GKE - expose service with Ingress and Internal Load Balancing

6/16/2021

I have REST API Web service on Internal GKE cluster which I would like to expose with internal HTTP load balancing.

Let's call this service "blue" service: I would like to expose it in following mapping:

http://api.xxx.yyy.internal/blue/isalive -> http://blue-service/isalive
http://api.xxx.yyy.internal/blue/v1/get -> http://blue-service/v1/get
http://api.xxx.yyy.internal/blue/v1/create -> http://blue-service/v1/create
http://api.xxx.yyy.internal/ -> http://blue-service/ (expose Swagger)

I'm omitting deployment yaml, since it's less relevant to discussion.

But my service yaml looks like this:

apiVersion: v1
kind: Service
metadata:
  name: blue-service
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: blue-service

My Ingress configuration is the following:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: blue-ingress
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: api.xxx.yyy.internal
    http:
      paths:
      - path: /blue/*
        backend:
          serviceName: blue-service
          servicePort: 80

However, I'm receiving 404 for all requests. /blue/v1/get, /blue/v1/create and /blue/isalive returns 404.

In my "blue" application I log all my notFound requests and I can clearly see that my URIs are not being rewritten, the requests hitting the application are /blue/v1/get, /blue/v1/create and /blue/isalive.

What am I missing in Ingress configuration? How can I fix those rewrites?

-- danny.lesnik
google-kubernetes-engine
kubernetes
kubernetes-ingress
nginx
url-rewriting

Similar Questions

How to install self signed certificate for Java in Docker/Kubernetes?
Rate limit requests based on HTTP header value on nginx-ingress
Installing kubernetes cluser on AWS using kubeadm returning error
is there something like httpPost in k8s
Google Kubernetes Engine - node upgrade procedure

1 Answer

6/17/2021

I solved the problem and writing it here to memo it and hopefully someone will find it as useful.

  • First problem is that I have mixed annotations types. one of GKE ingress controller and second for Nginx Server controller. Currently GKE ingress controller doesn't support URL rewrite feature, so I need to use nginx ingress controller.

  • so I need to install Nginx based ingress controller. It cloud be done easily using Helm chart or or deployment yaml. However, by default this controller will expose ingress using external load balancer and this not what I want. So we need to modify deployment charts or YAML file of this controller. I'm not using Helm, so I downoaded yaml itself using wget command.

    <pre><code> wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/cloud/deploy.yaml </code></pre>

Open it in editor and find the definition of Service names ingress-nginx-controller in namespace ingress-nginx. Add the following annotation.

cloud.google.com/load-balancer-type: "Internal"

After it I can run kubectl apply -f deploy.yaml command which will create Ingress controller for me. It will take a few minutes to provision it.

  • In addition I need to open firewall rule which will allow master nodes access worker nodes on port 8443/tcp.

  • And the last item is an ingress yaml itself which should look like this:

    <pre><code> apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: annotations: nginx.ingress.kubernetes.io/rewrite-target: /$2 kubernetes.io/ingress.class: "nginx" name: blue-ingress namespace: default spec: rules: - host: api.xxx.yyy.internal http: paths: - backend: serviceName: blue-service servicePort: 80 path: /blue(/|$)(.*)</code></pre>
-- danny.lesnik
Source: StackOverflow