502 for every other request ingress nginx controller

2/23/2020

I am having problem with ingress where every alternative requests return 502 Bad gateway.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'
spec:
  rules:
  - http:
      paths:
      - path: /test1/
        backend:
          serviceName: test1-service
          servicePort: 8089
      - path: /test2/
        backend:
          serviceName: test2-service
          servicePort: 4444 

--------------user-service.yaml------------

apiVersion: v1
kind: Service
metadata:
  name: user-service
spec:
  type: NodePort
  ports:
    - port: 8089
      targetPort: 8089
      nodePort: 31520
  selector:
    app: user

------------logs---------------

192.168.99.1 - - [23/Feb/2020:17:02:16 +0000] "GET /user/ HTTP/1.1" 200 13 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36" 427 0.002 [default-user-service-8089] [] 172.17.0.13:8089 13 0.002 200 5a8eb879aca5ebf682712c30d9f4c826
2020/02/23 17:02:22 [error] 2671#2671: *60232 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: _, request: "GET /user/ HTTP/1.1", upstream: "http://172.17.0.7:8089/user/", host: "192.168.99.101"
2020/02/23 17:02:22 [error] 2671#2671: *60232 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: _, request: "GET /user/ HTTP/1.1", upstream: "http://172.17.0.4:8089/user/", host: "192.168.99.101"
2020/02/23 17:02:22 [error] 2671#2671: *60232 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: _, request: "GET /user/ HTTP/1.1", upstream: "http://172.17.0.3:8089/user/", host: "192.168.99.101"
192.168.99.1 - - [23/Feb/2020:17:02:22 +0000] "GET /user/ HTTP/1.1" 502 565 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36" 453 0.005 [default-user-service-8089] [] 172.17.0.7:8089, 172.17.0.4:8089, 172.17.0.3:8089 0, 0, 0 0.001, 0.004, 0.000 502, 502, 502 4e37afefb17069ea66a46e06cb74fbd7
192.168.99.1 - - [23/Feb/2020:17:02:24 +0000] "GET /user/ HTTP/1.1" 200 13 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36" 453 0.001 [default-user-service-8089] [] 172.17.0.13:8089 13 0.000 200 3595217828be889af6f8f1508cefae59

No problems when accessing individual service using NodePort.

kubectl describe svc test1-service

Name:                     user-service
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 app=web
Type:                     NodePort
IP:                       xxxxxxxxxxx
Port:                     <unset>  8089/TCP
TargetPort:               8089/TCP
NodePort:                 <unset>  31520/TCP
Endpoints:                172.17.0.13:8089,172.17.0.3:8089,172.17.0.4:8089 + 1 more...
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>
-- Pram
docker
kubernetes
minikube

2 Answers

2/24/2020

I figured it out, I don't know the exact reason for this behavior, but there were label clash between deployments.

  labels:
    app: test1

I had not noticed that same label was use in other deployment also. After changing them issue is fixed, But not sure if it was directly related to the behavior, because for me every alternate request was failing.

-- Pram
Source: StackOverflow

2/23/2020

Try to remove the backslash and make sure that the services are running and listing to the configured ports

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'
spec:
  rules:
  - http:
      paths:
      - path: /test1
        backend:
          serviceName: test1-service
          servicePort: 8089
      - path: /test2
        backend:
          serviceName: test2-service
          servicePort: 4444
-- Al-waleed Shihadeh
Source: StackOverflow