GCE ingress with routes always falls back to default-http-backend

11/10/2016

Kubernetes version: 1.4.5

I have a very simple service with type: NodePort. It only returns some text on /info. I am using the default GKE ingress controller (the L7 Google load balancer) with TLS. If I use the following ingress everything works as expected:

Working ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: global-ingress
  namespace: global
  annotations:
    kubernetes.io/ingress.allow-http: "false"
spec:
  tls:
  - secretName: tls-secret
  backend:
    serviceName: gate-front
    servicePort: 80

curl -k https://130.211.39.140/info
POD: gate-front-1871107570-ue07p
IP: 10.0.2.26
REQ: /info

$ kubectl describe ing
Name:           global-ingress
Namespace:      global
Address:        130.211.39.140
Default backend:    gate-front:80 (10.0.2.25:8080,10.0.2.26:8080)
TLS:
  tls-secret terminates
Rules:
  Host  Path    Backends
  ----  ----    --------
  * *   gate-front:80 (10.0.2.25:8080,10.0.2.26:8080)
Annotations:
  backends:         {"k8s-be-31966--f3f0bf21d171a625":"HEALTHY"}
  https-forwarding-rule:    k8s-fws-global-global-ingress--f3f0bf21d171a625
  https-target-proxy:       k8s-tps-global-global-ingress--f3f0bf21d171a625
  url-map:          k8s-um-global-global-ingress--f3f0bf21d171a625

Broken ingress

However, if I introduce a rule and leave out the default backend, all requests return default backend - 404.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: global-ingress
  namespace: global
  annotations:
    kubernetes.io/ingress.allow-http: "false"
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /gate
        backend:
          serviceName: gate-front
          servicePort: 80

curl -k https://130.211.33.150/gate/info
default backend - 404

$ kubectl describe ing
Name:           global-ingress
Namespace:      global
Address:        130.211.33.150
Default backend:    default-http-backend:80 (10.0.2.3:8080)
TLS:
  tls-secret terminates
Rules:
  Host  Path    Backends
  ----  ----    --------
  *
        /gate   gate-front:80 (<none>)
Annotations:
  https-forwarding-rule:    k8s-fws-global-global2-ingress--f3f0bf21d171a625
  https-target-proxy:       k8s-tps-global-global2-ingress--f3f0bf21d171a625
  url-map:          k8s-um-global-global2-ingress--f3f0bf21d171a625
  backends:         {"k8s-be-31966--f3f0bf21d171a625":"HEALTHY","k8s-be-32552--f3f0bf21d171a625":"HEALTHY"}

If I add hosts and use curl -k --resolve ... I get the same behaviour.

I went through the following documentation and examples:

Can anyone shed some light on this?

-- Iulian
google-kubernetes-engine
kubernetes

2 Answers

11/10/2016

https://github.com/kubernetes/ingress-gce/blob/master/README.md#paths

Can you look over this part and comment if it solves the issue:

Note what just happened, the endpoint exposes /hostname, and the loadbalancer forwarded the entire matching url to the endpoint. This means if you had '/foo' in the Ingress and tried accessing /hostname, your endpoint would've received /foo/hostname and not known how to route it. Now update the Ingress to access static content via the /fs endpoint:

-- Prashanth B
Source: StackOverflow

9/15/2017

I was having a similar issue with a different cause, if you are using GCE just remember that every ingress controller gets a new IP.

I had pointed my DNS to the first ingress controller I made, and was not aware that new ingress controllers got new IPs. Traffic from the wrong host was being sent to my first ingress controller, so the 404 response was correct.

Ensure you point your DNS correctly by inspecting the ingress controller with kubectl:

kubectl describe ingress/<name>
-- Noah Huppert
Source: StackOverflow