Istio sidecar error Upstream connection failure in addition to 503 response code

3/9/2020

Istio sidecar for ldap server showing this error

[2020-03-09T10:26:28.802Z] "- - -" 0 - "-" "-" 2927 9389 120 - "-" "-" "-" "-" "192.168.64.17:8443" outbound|443||kubernetes.default.svc.cluster.local 172.17.0.22:35138 10.96.0.1:443 172.17.0.22:52526 - -
[2020-03-09T10:26:28.857Z] "- - -" 0 - "-" "-" 2921 1119287 65 - "-" "-" "-" "-" "192.168.64.17:8443" outbound|443||kubernetes.default.svc.cluster.local 172.17.0.22:35142 10.96.0.1:443 172.17.0.22:52530 - -
[2020-03-09T10:27:47.054Z] "- - -" 0 UF,URX "-" "-" 0 0 0 - "-" "-" "-" "-" "127.0.0.1:4444" inbound|4444|tcp-admin|opendj.default.svc.cluster.local - 172.17.0.22:4444 172.17.0.22:39246 - -
[2020-03-09T10:28:01.994Z] "- - -" 0 UF,URX "-" "-" 0 0 0 - "-" "-" "-" "-" "127.0.0.1:4444" inbound|4444|tcp-admin|opendj.default.svc.cluster.local - 172.17.0.22:4444 172.17.0.22:39520 - -
[2020-03-09T10:28:22.015Z] "- - -" 0 UF,URX "-" "-" 0 0 0 - "-" "-" "-" "-" "127.0.0.1:4444" inbound|4444|tcp-admin|opendj.default.svc.cluster.local - 172.17.0.22:4444 172.17.0.22:39860 - -
[2020-03-09T10:28:38.484Z] "- - -" 0 UF,URX "-" "-" 0 0 0 - "-" "-" "-" "-" "127.0.0.1:4444" inbound|4444|tcp-admin|opendj.default.svc.cluster.local - 172.17.0.22:4444 172.17.0.22:40148 - -

I understand that UF means Upstream connection failure in addition to 503 response code this error makes my ldap server not start up so my whole application fails. I have created a Gateway and VirtualService for the statefulset deployed application and shown below.

gate and vs

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: {{ include "opendj.name" . }}
  namespace: {{ .Release.Namespace }}
spec:
  hosts:
  - "*"
  gateways:
  - {{ include "opendj.name" . }} # can omit the namespace if gateway is in same namespace as virtual service.
  tcp:
  - match:
    - port: 1636
    route:
    - destination:
        host: opendj
        port:
          number: 1636
  - match:
    - port: 1389
    route:
    - destination:
        host: opendj
        port:
          number: 1389
  - match:
    - port: 4444
    route:
    - destination:
        host: opendj
        port:
          number: 4444
  - match:
    - port: 8989
    route:
    - destination:
        host: opendj
        port:
          number: 8989
---

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: {{ include "opendj.name" . }}
  namespace: {{ .Release.Namespace }}
spec:
  selector:
    app: my-gateway-controller
  servers:
  - port:
      number: 1636
      name: tcp-ldaps
      protocol: TCP
    hosts:
    - "*"
  - port:
      number: 1389
      name: tcp-ldap
      protocol: TCP
    hosts:
    - "*"
  - port:
      number: 8989
      name: tcp-repl
      protocol: TCP
    hosts:
    - "*"
  - port:
      number: 4444
      name: tcp-admin
      protocol: TCP
    hosts:
    - "*"

and the k8s service has all those ports defined.The error logs in ldap pod

INFO - entrypoint - 2020-03-09 11:34:50,698 - Installing OpenDJ.
WARNING - entrypoint - 2020-03-09 11:35:11,086 - Exception in thread "main" java.lang.IllegalArgumentException: Invalid network port provided: 0 is not included in the [1, 65535] range.
    at org.opends.server.types.HostPort.normalizePort(HostPort.java:362)
    at org.opends.server.types.HostPort.<init>(HostPort.java:218)
    at org.opends.quicksetup.UserData.setHostName(UserData.java:167)
    at org.opends.quicksetup.UserData.<init>(UserData.java:119)
    at org.opends.server.tools.InstallDS.execute(InstallDS.java:329)
    at org.opends.server.tools.InstallDS.mainCLI(InstallDS.java:268)
    at org.opends.server.tools.InstallDS.mainCLI(InstallDS.java:235)
    at org.opends.quicksetup.installer.SetupLauncher.launch(SetupLauncher.java:108)
    at org.opends.quicksetup.installer.SetupLauncher.main(SetupLauncher.java:55)

INFO - entrypoint - 2020-03-09 11:35:35,178 - Configuring OpenDJ.
WARNING - entrypoint - 2020-03-09 11:35:50,882 - 
Unable to connect to the server at
"test5-opendj-0.opendj.default.svc.cluster.local" on port 4444

NOTE: I am deploying this on minikube with istio using nodePort

-- Shammir
istio
kubernetes

0 Answers