Can someone please help to spot the issue with ingress-2 ingress rule? why ingress-1 is working vs ingress-2 is not working.
Description of my setup, I have two deployments:
1st deployment is of nginx
2nd deployment is of httpd
Both of the deployments are exposed via ClusterIP services named nginx-svc and httpd-svc respectively. All the endpoints are proper for the services. However, while
setting up the ingress for these services, I am not able to setup the ingress using host (as described in ingress-2). however, when I am using ingress-1, things work fine.
// my host file for name resolution
grep myapp.com /etc/hosts
127.0.0.1 myapp.com// deployment details
kubectl get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 29m
httpd 3/3 3 3 29m// service details
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 7h48m
nginx-svc ClusterIP 10.152.183.233 <none> 80/TCP 28m
httpd-svc ClusterIP 10.152.183.58 <none> 80/TCP 27m// endpoints details
kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.2.15:16443 7h51m
nginx-svc 10.1.198.86:80,10.1.198.87:80,10.1.198.88:80 31m
httpd-svc 10.1.198.89:80,10.1.198.90:80,10.1.198.91:80 31mAttempt-1: ingress-1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-1
spec:
rules:
- http:
paths:
- path: /nginx
pathType: Prefix
backend:
service:
name: nginx-svc
port:
number: 80
- path: /httpd
pathType: Prefix
backend:
service:
name: httpd-svc
port:
number: 80// Example showing that ingress routing is working fine when ingress-1 is used:
curl myapp.com/nginx
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
curl myapp.com/httpd
<html><body><h1>It works!</h1></body></html>// following ingress rule is not working as I was expecting
Attempt-2: ingress-2
kind: Ingress
metadata:
name: ingress-2
spec:
rules:
- host: "myapp.com"
http:
paths:
- pathType: Prefix
path: "/nginx"
backend:
service:
name: nginx-svc
port:
number: 80
- pathType: Prefix
path: "/httpd"
backend:
service:
name: httpd-svc
port:
number: 80// I could not spot any issue in the ing describe
kubectl describe ingress ingress-2
Name: ingress-2
Namespace: default
Address: 127.0.0.1
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
myapp.com
/nginx nginx-svc:80 (10.1.198.86:80,10.1.198.87:80,10.1.198.88:80)
/httpd httpd-svc:80 (10.1.198.89:80,10.1.198.90:80,10.1.198.91:80)
Annotations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 9m15s (x2 over 10m) nginx-ingress-controller Scheduled for sync// example showing ingress routing is not working with this ingress resource
curl myapp.com/nginx
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.21.1</center>
</body>
</html>
curl myapp.com/httpd
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
</body></html>I created a one node microk8s cluster following official documentation and I wasn't able to reproduce behaviour you described which is correct behaviour. Added two pods with mendhak/http-https-echo image (highly recommend: very convenient for troubleshooting ingress or understanding how ingress works) and two services for each of pods.
The difference between two ingress rules is first ingress rule listens on all domains (HOSTS):
$ mkctl get ing -o wide
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-1 public * 127.0.0.1 80 2m53s
$ curl -I --header "Host: myapp.com" http://127.0.0.1/httpd
HTTP/1.1 200 OK
$ curl -I --header "Host: example.com" http://127.0.0.1/httpd
HTTP/1.1 200 OK
$ curl -I --header "Host: myapp.com" http://127.0.0.1/missing_url
HTTP/1.1 404 Not FoundWhile the second ingress rule will serve only myapp.com domain (HOST):
$ mkctl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-2 public myapp.com 127.0.0.1 80 60s
$ curl -I --header "Host: myapp.com" http://127.0.0.1/httpd
HTTP/1.1 200 OK
$ curl -I --header "Host: example.com" http://127.0.0.1/httpd
HTTP/1.1 404 Not FoundLast results in your question actually show that ingress is working as expected. You're getting responses not from kubernetes ingress but from pods within the cluster. First response is 404 from nginx 1.21.0 and second is 404 from apache.
This happens because ingress sends requests to pods with the same path from URL without any transformations. For instance (this output I got using image mentioned above):
$ curl myapp.com/httpd
{
"path": "/httpd"
...
While both nginx and apache are serving on /.
Nginx ingress has a lot of features and one of them is rewriting which helps to transform paths from what ingress gets to what goes to pods.
For example, if request goes to http://myapp.com/nginx then it will be directed to nginx service with /nginx path which will cause nginx to throw 404 since there's nothing on this path.
Ingress rule below fixes this by adding rewrite-target to / which we need to pass to nginx and apache services:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-2
annotations:
# kubernetes.io/ingress.class: nginx # this should be uncommented if ingress used in "regular" cluster
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: myapp.com
http:
paths:
- path: /nginx
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /httpd
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80Quick test how it works:
$ curl myapp.com/nginx
{
"path": "/",
...
And
$ curl myapp.com/httpd
{
"path": "/",
...
As you can see now path is /.
Switching image to nginx and:
$ curl myapp.com/nginx
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
...