Kubernetes Ingress load balancer rewrites everything to index; why?

10/12/2017

TL;DR for some reason every http service seems to have the path rewritten as /

I'm pretty new to Kubernetes and trying to set up an ingress load balancer.

Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.6", GitCommit:"4bc5e7f9a6c25dc4c03d4d656f2cefd21540e28c", GitTreeState:"clean", BuildDate:"2017-09-14T06:55:55Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7+", GitVersion:"v1.7.6-gke.1", GitCommit:"407dbfe965f3de06b332cc22d2eb1ca07fb4d3fb", GitTreeState:"clean", BuildDate:"2017-09-27T21:21:34Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

I'm setting everything up in Google Container Engine (GKE)

It's partially working, which is a good thing. However, for some reason it's routing everything to my service as if the request is directed at /.

What's happening here? My guess is poor configuration.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: main-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "ingress-main-ip"
spec:
  tls:
  - secretName: cert-main
    hosts:
    - api.example.com
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /my_service/*
        backend:
          serviceName: myService
          servicePort: main-port
      - path: /my_service2/*
        backend:
          serviceName: myService2
          servicePort: main-port

Whenever I make a request to api.example.com/my_service/ then myService will register the request as coming from index--no problem.

Same thing with https://api.example.com/my_service2/ which goes to myService2

But whenever I make a request to https://api.example.com/my_service/bogus/path/that/should/return/404 then my service seems to think the request is being directed at the / path again just as before.

Of course when I run myService locally (which happens to be a Go http server), then it works perfectly fine. To be especially clear: running this locally 127.0.0.1/bogus/path/that/should/return/404 returns a 404 as expected (and of course other API endpoints also work)

Here's an extremely strange thing: whenever I visit the static IP address directly (aka ingress-main-ip, let's just say it's 1.2.3.4), for example http://1.2.3.4/my_service/ or http://1.2.3.4/my_service2/ it always returns a 404 from the GKE default backend, so my services aren't even getting routed to.

Summary:

  1. Why are the domain versions of the requests all routing to /
  2. Why do the direct IP requests not work at all?

Thank you for your time and help!

-- Wyatt Arent
gcp
google-kubernetes-engine
kubernetes

2 Answers

10/13/2017

I will answer the 2nd question.

The extremely strange thing is due to the HTTP header "Host" matters because you specified it in the rules section:

rules:
  - host: api.example.com           <----- THIS
    http:
      paths:
      - path: /my_service/*
        backend:
          serviceName: myService
          servicePort: main-port
      - path: /my_service2/*
        backend:
          serviceName: myService2
          servicePort: main-port

So it's used to route the request to the proper service. If your service controller is nginx, so that "host:" is its server_name.

-- Robert
Source: StackOverflow

10/13/2017

Robert is right about the second question (Many thanks)

I just figured out the main question, why all my paths seemed to be rewritten to /

It turns out that the path is sent to the server with the "path" as specified in the ingress configuration to the actual service. So for example, when trying to do a GET request to api.example.com/my_service/my_endpoint, I was expecting my_service to just get the path /my_endpoint but it was actually getting the full /my_service/my_endpoint

This seems like a pretty bad kubernetes side effect in my opinion, and the path should really be a relative rewrite.

In any case it's fixed now. The reason I "thought" that they were all being rewritten to simply / is because for some reason (logic errors I wrote into my code) my multiplexer was capturing every unknown request and sending it to my index controller. Go figure!

-- Wyatt Arent
Source: StackOverflow