Unable to configure path based routing in Kubernetes Ingress

4/16/2018

I am running a 3 nodes bare metal cluster on version 1.9.5.

IPs of the 3 nodes are :

[root@node1 new]# kubectl get nodes
NAME      STATUS    ROLES         AGE       VERSION    IP
node1     Ready     master,node   1d        v1.9.5     172.16.16.1
node2     Ready     node          1d        v1.9.5     172.16.16.2
node3     Ready     node          1d        v1.9.5     172.16.16.3

Everything I explain below is being done in 1 namespace i.e. ingress-nginx

I have 2 apps deployed.

[root@node1 new]# kubectl get po -n ingress-nginx
NAME                                     READY     STATUS    RESTARTS   AGE
app1-5d4d466cc7-595lc                    1/1       Running   0          25m
app2-55cf97d86d-9v8gl                    1/1       Running   0          25m

Their services are :

[root@node1 new]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                        AGE
appsvc1                NodePort    10.233.60.145   <none>        80:32601/TCP                   25m
appsvc2                NodePort    10.233.46.230   <none>        80:30616/TCP                   25m

So when I access them via NodePort, I get my desired result.

curl http://172.16.16.2:32601
<h1>Hello app1!</h1>

curl http://172.16.16.2:30616
<h1>Hello app2!</h1>

Now my aim is to configure path based routing using nginx ingress controller so that at the end of it, I can get routing to app1 using

curl http://172.16.16.2/app1
<h1>Hello app1!</h1>

&

curl http://172.16.16.2/app2
<h1>Hello app2!</h1>

So now I have setup an ingress controller using ingress-nginx .

The nginx controller is also deployed in the same namespace at the apps i.e. ingress-nginx

My ingress controller is successfully deployed as the logs indicate :

[root@node1 new]# kubectl logs nginx-ingress-controller-9c7b694-bjn6h -n ingress-nginx
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:    0.12.0
  Build:      git-1df421a
  Repository: https://github.com/kubernetes/ingress-nginx
-------------------------------------------------------------------------------

I0415 16:08:18.736790       5 main.go:225] Running in Kubernetes Cluster version v1.9 (v1.9.5) - git (clean) commit f01a2bf98249a4db383560443a59bed0c13575df - platform linux/amd64
I0415 16:08:18.743855       5 main.go:84] validated ingress-nginx/default-http-backend as the default backend
I0415 16:08:19.182913       5 stat_collector.go:77] starting new nginx stats collector for Ingress controller running in namespace  (class nginx)
I0415 16:08:19.182944       5 stat_collector.go:78] collector extracting information from port 18080
I0415 16:08:19.211749       5 nginx.go:281] starting Ingress controller
I0415 16:08:20.325839       5 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"app-ingress", UID:"918405ac-410e-11e8-a473-080027917402", APIVersion:"extensions", ResourceVersion:"75539", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/app-ingress
I0415 16:08:20.326503       5 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"ingress-nginx", Name:"nginx-ingress", UID:"8dc22500-410e-11e8-a473-080027917402", APIVersion:"extensions", ResourceVersion:"75538", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress ingress-nginx/nginx-ingress
I0415 16:08:20.413514       5 store.go:614] running initial sync of secrets
I0415 16:08:20.413591       5 nginx.go:302] starting NGINX process...
I0415 16:08:20.418356       5 leaderelection.go:174] attempting to acquire leader lease...
W0415 16:08:20.422596       5 controller.go:777] service ingress-nginx/nginx-ingress does not have any active endpoints
I0415 16:08:20.422686       5 controller.go:183] backend reload required
I0415 16:08:20.422694       5 stat_collector.go:34] changing prometheus collector from  to default
I0415 16:08:20.439620       5 status.go:196] new leader elected: nginx-ingress-controller-9c7b694-h2n4b
I0415 16:08:20.534277       5 controller.go:192] ingress backend successfully reloaded...
W0415 16:08:28.768140       5 controller.go:777] service ingress-nginx/nginx-ingress does not have any active endpoints
I0415 16:09:00.478068       5 leaderelection.go:184] successfully acquired lease ingress-nginx/ingress-controller-leader-nginx
I0415 16:09:00.478207       5 status.go:196] new leader elected: nginx-ingress-controller-9c7b694-bjn6h

Then I configured my ingress using :

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: app-ingress
spec:
  rules:
  - host: my-test.com
    http:
      paths:
      - backend:
          serviceName: appsvc1
          servicePort: 80
        path: /app1
      - backend:
          serviceName: appsvc2
          servicePort: 80
        path: /app2

I created this using :

kubectl create -f app-ingress.yaml -n ingress-nginx

I then expose this using :

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress
spec:
  type: NodePort
  externalIps:
  - 172.16.16.1
  - 172.16.16.2
  - 172.16.16.3
  ports:
    - port: 80
      nodePort: 30000
      name: http
  selector:
    app: nginx-ingress

app: nginx-ingress points to label from my label in ingress-controller pod.

I deploy it using :

kubectl create -f nginx-ingress-controller-service.yaml -n=ingress

But when I try to access the apps using the URLs, I get :

curl http://172.16.16.2/app1
default backend - 404

&

curl http://172.16.16.2/app2
default backend - 404

Even doing

curl http://my-test.com/app1
default backend - 404

&

curl http://my-test.com/app2
default backend - 404

does not work.

My /etc/hosts files is :

172.16.16.1 my-test.com

Is there something I have missed or am doing wrong ?

[root@node1 new]# kubectl get all -n ingress-nginx
NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/app1                       2         2         2            2           48m
deploy/app2                       2         2         2            2           48m
deploy/default-http-backend       1         1         1            1           3h
deploy/nginx-ingress-controller   1         1         1            1           3h

NAME                                  DESIRED   CURRENT   READY     AGE
rs/app1-5d4d466cc7                    2         2         2         48m
rs/app2-55cf97d86d                    2         2         2         48m
rs/default-http-backend-55c6c69b88    1         1         1         3h
rs/nginx-ingress-controller-9c7b694   1         1         1         3h

NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/app1                       2         2         2            2           48m
deploy/app2                       2         2         2            2           48m
deploy/default-http-backend       1         1         1            1           3h
deploy/nginx-ingress-controller   1         1         1            1           3h

NAME                                  DESIRED   CURRENT   READY     AGE
rs/app1-5d4d466cc7                    2         2         2         48m
rs/app2-55cf97d86d                    2         2         2         48m
rs/default-http-backend-55c6c69b88    1         1         1         3h
rs/nginx-ingress-controller-9c7b694   1         1         1         3h

NAME                                        READY     STATUS    RESTARTS   AGE
po/app1-5d4d466cc7-595lc                    1/1       Running   0          48m
po/app1-5d4d466cc7-5dn72                    1/1       Running   0          48m
po/app2-55cf97d86d-9v8gl                    1/1       Running   0          48m
po/app2-55cf97d86d-lckpn                    1/1       Running   0          48m
po/default-http-backend-55c6c69b88-8shkt    1/1       Running   0          3h
po/nginx-ingress-controller-9c7b694-bjn6h   1/1       Running   0          52m

NAME                       TYPE        CLUSTER-IP      EXTERNAL-IP                           PORT(S)                      AGE
svc/appsvc1                NodePort    10.233.60.145   <none>                                80:32601/TCP                 48m
svc/appsvc2                NodePort    10.233.46.230   <none>                                80:30616/TCP                 48m
svc/default-http-backend   ClusterIP   10.233.5.30     <none>                                80/TCP                       3h
svc/ingress-nginx          NodePort    10.233.6.186    <none>                                80:31301/TCP,443:32103/TCP   3h
svc/nginx-ingress          NodePort    10.233.9.163    172.16.16.1,172.16.16.2,172.16.16.3   80:30000/TCP                 2h

I even changed my ingress config to remove the host and be open for all hosts:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: app-ingress
spec:
  rules:
  - http:
      paths:
      - path: /app1
        backend:
          serviceName: appsvc1
          servicePort: 80
      - path: /app2
        backend:
          serviceName: appsvc2
          servicePort: 80

And now I get redirected.

[root@node1 ~]# curl http://172.16.16.1/app1
<html>
<head><title>308 Permanent Redirect</title></head>
<body bgcolor="white">
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.13.9</center>
</body>
</html>
[root@node1 ~]# curl http://172.16.16.1/app2
<html>
<head><title>308 Permanent Redirect</title></head>
<body bgcolor="white">
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.13.9</center>
</body>
</html>

Ingress controller logs say :

10.233.102.128 - [10.233.102.128] - - [15/Apr/2018:17:35:52 +0000] "GET /app1 HTTP/1.1" 308 187 "-" "curl/7.29.0" 80 0.000 [ingress-nginx-appsvc1-80] - - - -
10.233.102.128 - [10.233.102.128] - - [15/Apr/2018:17:36:09 +0000] "GET /app1 HTTP/1.1" 308 187 "-" "curl/7.29.0" 79 0.000 [ingress-nginx-appsvc1-80] - - - -
10.233.102.128 - [10.233.102.128] - - [15/Apr/2018:17:36:11 +0000] "GET /app2 HTTP/1.1" 308 187 "-" "curl/7.29.0" 79 0.000 [ingress-nginx-appsvc2-80] - - - -
10.233.102.128 - [10.233.102.128] - - [15/Apr/2018:17:36:25 +0000] "GET /app2 HTTP/1.1" 308 187 "-" "curl/7.29.0" 85 0.000 [ingress-nginx-appsvc2-80] - - - -
10.233.102.128 - [10.233.102.128] - - [15/Apr/2018:17:36:51 +0000] "GET /app2 HTTP/1.1" 308 187 "-" "curl/7.29.0" 80 0.000 [ingress-nginx-appsvc2-80] - - - -
--
kubernetes
kubernetes-ingress
nginx

2 Answers

5/26/2020

You will have to set use-forwarded-headers: "true" in your nginx controller configmap and use the kubernetes.io/ingress.class: "nginx" annotation in your ingress. That should get it working

-- Snigdha Sambit Aryakumar
Source: StackOverflow

4/19/2018

All your configs are fine, except one thing related to annotation:

  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /

You don't need this annotation, because it's processed earlier than routing and you always have the request to / . You have no route for this path, so just remove this annotation and it should work.

-- Nick Rak
Source: StackOverflow