Routing troubleshooting with Kubernetes Ingress

9/25/2019

I tried to setup a GKE environment with a frontend pod (cup-fe) and a backend one, used to authenticate the user upon login (cup-auth), but I can't get my ingress to work.

Following is the frontend pod (cup-fe) running nginx with an angular app. I created also a static IP address resolved by "cup.xxx.it" and "cup-auth.xxx.it" dns:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cup-fe
  namespace: default
  labels:
    app: cup-fe
spec:
  replicas: 2
  selector:
    matchLabels:
      app: "cup-fe"
  template:
    metadata:
      labels:
        app: "cup-fe"
    spec:
      containers:
      - image: "eu.gcr.io/xxx-cup-yyyyyy/cup-fe:latest"
        name: "cup-fe"
      dnsPolicy: ClusterFirst

Then is the auth pod (cup-auth):

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cup-auth
  namespace: default
  labels:
    app: cup-auth
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cup-auth
  template:
    metadata:
      labels:
        app: cup-auth
    spec:
      containers:
        image:  "eu.gcr.io/xxx-cup-yyyyyy/cup-auth:latest"
        imagePullPolicy: Always
        name: cup-auth
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 8443
          protocol: TCP
        - containerPort: 8778
          name: jolokia
          protocol: TCP
        - containerPort: 8888
          name: management
          protocol: TCP
      dnsPolicy: ClusterFirst

Then I created two NodePorts to expose the above pods:

kubectl expose deployment cup-fe --type=NodePort --port=80
kubectl expose deployment cup-auth --type=NodePort --port=8080

Last, I created an ingress to route external http requests towards services:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: http-ingress
  namespace: default
  labels:
    app: http-ingress
spec:
  rules:
  - host: cup.xxx.it
    http:
      paths:
      - path: /*
        backend:
          serviceName: cup-fe
          servicePort: 80
  - host: cup-auth.xxx.it
    http:
      paths:
      - path: /*
        backend:
          serviceName: cup-auth

So, I can reach the frontend pod at http://cup.xxx.it, the angular app redirects me to http://cup-auth.xxx.it/login, but I get only 502 bad request. With kubectl describe ingress command, I can see an unhealthy backend for the cup-auth pod.

I paste a successful output by using cup-auth label:

$ kubectl exec -it cup-fe-7f979bb747-6lqfx wget cup.xxx.it/login
Connecting to cup.xxx.it
login                100% |********************************|  1646  0:00:00 ETA

And then the not working output:

$ kubectl exec -it cup-fe-7f979bb747-6lqfx wget cup-auth.xxx.it/login
Connecting to cup-auth.xxx.it
wget: server returned error: HTTP/1.1 502 Bad Gateway
command terminated with exit code 1
-- duns
google-kubernetes-engine
kubernetes
kubernetes-ingress

1 Answer

10/29/2019

I tried and replicated your setup as much as I could, but did not have any issues. I can call the cup-auth.testdomain.internal/login normally within and outside the pods.

Usually, the 502 errors occur when the request received to the LB couldn't forward to a backend. Since you mention that you are seeing an unhealthy backend this can be the reason.

This could be due to a wrong configuration of the health checks or a problem with your application.

First I would look at the logs to see the reason the request is failing, and eliminate that there is no issue with the health checks or with the application itself.

-- Kostikas Visnia
Source: StackOverflow