How do I know if the 404 error returned by the ingress controller will resolve itself by waiting?

11/20/2019

I create my ingress. Wait for a few minutes. Then I get a HTTP/1.1 404 Not Found error when I send an HTTP reqeust to it.

However, if I wait for a few more minutes and send the exact same request, everything works.

My question is: Aside from waiting, is there anything that I can do to know for sure if the error I am getting is due to a misconfiguration vs just having to wait for k8s to do its thing?

Here is the describe after the request failed:

    >kubectl describe ing ingress-monza-predictors
Name:             ingress-monza-predictors
Namespace:        default
Address:          35.244.250.224
Default backend:  default-http-backend:80 (10.40.2.10:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
        /seatbelt-driver      predictor-classification-seatbelt-driver-service-node-port:4444 (<none>)
        /seatbelt-passenger   predictor-classification-seatbelt-passenger-service-node-port:4444 (<none>)
Annotations:
  ingress.kubernetes.io/backends:                    {"k8s-be-30355--62f4c45447b62142":"Unknown","k8s-be-31413--62f4c45447b62142":"Unknown","k8s-be-32727--62f4c45447b62142":"Unknown"}
  ingress.kubernetes.io/forwarding-rule:             k8s-fw-default-ingress-monza-predictors--62f4c45447b62142
  ingress.kubernetes.io/target-proxy:                k8s-tp-default-ingress-monza-predictors--62f4c45447b62142
  ingress.kubernetes.io/url-map:                     k8s-um-default-ingress-monza-predictors--62f4c45447b62142
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.global-static-ip-name":"ingress-monza-predictors"},"name":"ingress-monza-predictors","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"predictor-classification-seatbelt-driver-service-node-port","servicePort":4444},"path":"/seatbelt-driver"},{"backend":{"serviceName":"predictor-classification-seatbelt-passenger-service-node-port","servicePort":4444},"path":"/seatbelt-passenger"}]}}]}}

  kubernetes.io/ingress.global-static-ip-name:  ingress-monza-predictors
Events:
  Type    Reason  Age   From                     Message
  ----    ------  ----  ----                     -------
  Normal  ADD     96s   loadbalancer-controller  default/ingress-monza-predictors
  Normal  CREATE  22s   loadbalancer-controller  ip: 35.244.250.224

Here is the describe after the request succeeded (a few mins later):

    >kubectl describe ing ingress-monza-predictors
Name:             ingress-monza-predictors
Namespace:        default
Address:          35.244.250.224
Default backend:  default-http-backend:80 (10.40.2.10:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
        /seatbelt-driver      predictor-classification-seatbelt-driver-service-node-port:4444 (<none>)
        /seatbelt-passenger   predictor-classification-seatbelt-passenger-service-node-port:4444 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.global-static-ip-name":"ingress-monza-predictors"},"name":"ingress-monza-predictors","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"predictor-classification-seatbelt-driver-service-node-port","servicePort":4444},"path":"/seatbelt-driver"},{"backend":{"serviceName":"predictor-classification-seatbelt-passenger-service-node-port","servicePort":4444},"path":"/seatbelt-passenger"}]}}]}}

  kubernetes.io/ingress.global-static-ip-name:  ingress-monza-predictors
  ingress.kubernetes.io/backends:               {"k8s-be-30355--62f4c45447b62142":"HEALTHY","k8s-be-31413--62f4c45447b62142":"HEALTHY","k8s-be-32727--62f4c45447b62142":"HEALTHY"}
  ingress.kubernetes.io/forwarding-rule:        k8s-fw-default-ingress-monza-predictors--62f4c45447b62142
  ingress.kubernetes.io/target-proxy:           k8s-tp-default-ingress-monza-predictors--62f4c45447b62142
  ingress.kubernetes.io/url-map:                k8s-um-default-ingress-monza-predictors--62f4c45447b62142
Events:
  Type    Reason  Age   From                     Message
  ----    ------  ----  ----                     -------
  Normal  ADD     10m   loadbalancer-controller  default/ingress-monza-predictors
  Normal  CREATE  9m5s  loadbalancer-controller  ip: 35.244.250.224

My hunch is that if the backends show an "Unknown" status, I should just wait. However, if the backends show a "HEALTHY" status, then the problem is probably due to a misconfiguration. Is this correct?

-- Alex Ryan
google-kubernetes-engine

1 Answer

11/20/2019

That’s right. According to this article, it will take a few minutes for GKE to allocate an external IP address and set up forwarding rules until the load balancer is ready to serve your application.

-- Wilfred L.
Source: StackOverflow