Kubernetes AKS - Ingress Backend 404 error

3/24/2020

Im using aks on azure. I have 404 when Im trying to request my service.

Ingress:

Name:             portal-deployment2
Namespace:        portal-api
Address:          10.226.16.4
Default backend:  default-http-backend:80 (<none>)
TLS:
  portal-certificate terminates portal.example.com
Rules:
  Host                                      Path  Backends
  ----                                      ----  --------
  portal.example.com
                                            /   portal-deployment2:80 (10.226.16.48:9999)
Annotations:
  certmanager.k8s.io/acme-challenge-type:  http01
  certmanager.k8s.io/cluster-issuer:       letsencrypt-prod
  kuberentes.io/tls-acme:                  true
  kubernetes.io/ingress.class:             nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  11m   nginx-ingress-controller  Ingress portal-api/portal-deployment2
  Normal  UPDATE  10m   nginx-ingress-controller  Ingress portal-api/portal-deployment2

Service:

Name:              portal-deployment2
Namespace:         portal-api
Labels:            app.kubernetes.io/instance=portal
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=deployment2
                   app.kubernetes.io/version=1.16.0
                   helm.sh/chart=deployment2-0.1.0
Annotations:       <none>
Selector:          app.kubernetes.io/instance=portal,app.kubernetes.io/name=deployment2
Type:              ClusterIP
IP:                10.0.187.224
Port:              http  80/TCP
TargetPort:        9999/TCP
Endpoints:         10.226.16.48:9999
Session Affinity:  None
Events:            <none>

When I use nginx-controller bash:

bash-5.0$ curl 10.226.16.48:9999
{"timestamp":"2020-03-24T09:39:34.545+0000","status":404,"error":"Not Found","message":"No message available","path":"/"}

bash-5.0$ curl 10.0.187.224
{"timestamp":"2020-03-24T09:39:43.565+0000","status":404,"error":"Not Found","message":"No message available","path":"/"}

bash-5.0$ curl -k portal.example.com
default backend - 404

It looks like nginx controller can't handle the route and redirect me to appropriate POD

-- TeK
azure-aks
kubernetes
kubernetes-ingress
nginx-ingress

1 Answer

3/27/2020

After reading the SO post, I think that the issue could be related to the DNS entry related to portal.example.com

Since the User shows that curling the POD/Service IP are returning expected results but when Trying using the DNS name, I would assume that the DNS resolution of portal.example.com isn't returning the IP that the User is expecting.

There are many reason why this wouldn't work,

1) the DNS entry isn't managed automatically by a 3rd party service like: external-DNS 2) the DNS entry wasn't added manually to a DNS zone (example.com) 3) the DNS entry isn't properly overwritten on the user's machine /etc/host file.

It should note that portal.example.com needs to resolve to the service IP of the NGINX ingress controller Kubernetes Service IP

Hoping this help and clarifies the potential issue.

-- Joe Mellin
Source: StackOverflow