K
Q

IstIO egress gateway gives HTTP 503 error

12/4/2019

I have the following manifest for deploying a IstIO egress gateway routing:

---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: REDACTED-egress-se
spec:
  hosts:
  - sahfpxa.REDACTED
  ports:
  - number: 8080
    name: http-port
    protocol: HTTP
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: sahfpxa-REDACTED-egress-gw
spec:
  selector:
    istio: egressgateway
  servers:
  - port:
      number: 8080
      name: http
      protocol: HTTP
    hosts:
    - sahfpxa.REDACTED
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: egressgateway-for-sahfpxa-REDACTED
spec:
  host: istio-egressgateway.istio-system.svc.cluster.local
  subsets:
  - name: sahfpxa
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-sahfpxa-REDACTED-through-egress-gateway
spec:
  hosts:
  - sahfpxa.REDACTED
  gateways:
  - REDACTED/REDACTED-egress-gw
  - mesh
  http:
  - match:
    - gateways:
      - mesh
      port: 8080
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        subset: sahfpxa
        port:
          number: 80
      weight: 100
  - match:
    - gateways:
      - REDACTED/sahfpxa-REDACTED-egress-gw
      port: 8080
    route:
    - destination:
        host: sahfpxa.REDACTED
        port:
          number: 8080
      weight: 100

But I get a connection refused from the sidecar istio-proxy container Pod of the affected namespace and a HTTP 503 error from the workload container in that namespace.

Any ideas what could be wrong with the configuration or how I can debug it?

Thanks in advance.

Best regards, rforberger

-- Ronny Forberger
istio
kubernetes

Similar Questions

kubernetes dashboard unauthorized using Authorization: Bearer header
Can Openshift templates create objects over multiple projects
nginx-ingress return default backend on https request
kubectl port-forward can be dangerous used in prod environment?
Install two separate KongHQ in same kube deployment

1 Answer

12/9/2019

There were few errors in Your deployment manifest like DestinationRule was not pointing at Your ServiceEntry.

You can try to match Yours with these manifest files:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: etth
spec:
  hosts:
  - etth.pl
  ports:
  - number: 8080
    name: http-port
    protocol: HTTP
  resolution: DNS
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-egressgateway
spec:
  selector:
    istio: egressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - etth.pl
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: egressgateway-for-cnn
spec:
  host: istio-egressgateway.istio-system.svc.cluster.local
  subsets:
  - name: etth
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-cnn-through-egress-gateway
spec:
  hosts:
  - etth.pl
  gateways:
  - istio-egressgateway
  - mesh
  http:
  - match:
    - gateways:
      - mesh
      port: 80
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        subset: etth
        port:
          number: 80
      weight: 100
  - match:
    - gateways:
      - istio-egressgateway
      port: 80
    route:
    - destination:
        host: etth.pl
        port:
          number: 8080
      weight: 100

You can check if routes are present in: istioctl pc routes $(kubectl get pods -l istio=egressgateway -o jsonpath='{.items[0].metadata.name}' -n istio-system).istio-system -o json

$ istioctl pc routes $(kubectl get pods -l istio=egressgateway -o jsonpath='{.items[0].metadata.name}' -n istio-system).istio-system -o json
[
    {
        "name": "http.80",
        "virtualHosts": [
            {
                "name": "etth.pl:80",
                "domains": [
                    "etth.pl",
                    "etth.pl:80"
                ],
                "routes": [
                    {
                        "match": {
                            "prefix": "/",
                            "caseSensitive": true
                        },
                        "route": {
                            "cluster": "outbound|8080||etth.pl",
                            "timeout": "0s",
                            "retryPolicy": {
                                "retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
                                "numRetries": 2,
                                "retryHostPredicate": [
                                    {
                                        "name": "envoy.retry_host_predicates.previous_hosts"
                                    }
                                ],
                                "hostSelectionRetryMaxAttempts": "5",
                                "retriableStatusCodes": [
                                    503
                                ]
                            },
                            "maxGrpcTimeout": "0s"
                        },
                        "metadata": {
                            "filterMetadata": {
                                "istio": {
                                    "config": "/apis/networking/v1alpha3/namespaces/default/virtual-service/direct-cnn-through-egress-gateway"
                                }
                            }
                        },
                        "decorator": {
                            "operation": "etth.pl:8080/*"
                        },
                        "typedPerFilterConfig": {
                            "mixer": {
                                "@type": "type.googleapis.com/istio.mixer.v1.config.client.ServiceConfig",
                                "disableCheckCalls": true,
                                "mixerAttributes": {
                                    "attributes": {
                                        "destination.service.host": {
                                            "stringValue": "etth.pl"
                                        },
                                        "destination.service.name": {
                                            "stringValue": "etth.pl"
                                        },
                                        "destination.service.namespace": {
                                            "stringValue": "default"
                                        }
                                    }
                                },
                                "forwardAttributes": {
                                    "attributes": {
                                        "destination.service.host": {
                                            "stringValue": "etth.pl"
                                        },
                                        "destination.service.name": {
                                            "stringValue": "etth.pl"
                                        },
                                        "destination.service.namespace": {
                                            "stringValue": "default"
                                        }
                                    }
                                }
                            }
                        }
                    }
                ]
            }
        ],
        "validateClusters": false
    },
    {
        "virtualHosts": [
            {
                "name": "backend",
                "domains": [
                    "*"
                ],
                "routes": [
                    {
                        "match": {
                            "prefix": "/stats/prometheus"
                        },
                        "route": {
                            "cluster": "prometheus_stats"
                        }
                    }
                ]
            }
        ]
    }
]
-- Piotr Malec
Source: StackOverflow