Istio throwing 404 for URL mapped in Virtual Service

7/31/2019

Cloud Provider: Azure Istio Version: 1.19 Kubernetes Version: 1.12.8

Below is my Virtual Service and Gateway yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: app-ingress-rules
  namespace: istio-system
# Define the routes less restrictive first to more restrictive. this file gets read from bottom to top.
spec:
  # https://istio.io/docs/reference/config/istio.networking.v1alpha3/#VirtualService
  gateways: # The default `mesh` value for when left blank is doesn't seem to propigate the rule properly. For now, always use a list of FQDN gateways
    - tls-gateway
  hosts:
    - "dev.myapp.westus.cld.something.com" # This host is part of installed HTTPS cert
    - "nonprod.westus.cloudapp.azure.com" # This host is NOT part of installed HTTPS cert
  http:
  # Routes
  - match:
    - uri:
        prefix: /myapp/status
    rewrite:
      uri: /status
    route:
    - destination:
        host: myapp.myapp-ns.svc.cluster.local  
--------------------------------------------
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: tls-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"

If I do

curl -Iv https://dev.myapp.westus.cld.something.com/myapp/status

I get HTTP 404 as below

    *   Trying 123.12.123.12...
* TCP_NODELAY set
* Connected to dev.myapp.westus.cld.something.com (123.12.123.12.) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=***; L=*****; O=******, Inc.; CN=*********
*  start date: Jul 26 20:32:59 2019 GMT
*  expire date: Jul 26 20:32:59 2021 GM
*  subjectAltName: host "dev.myapp.westus.cld.something.com" matched cert's "dev.myapp.westus.cld.something.com"
*  issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign RSA OV SSL CA 2018
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f995e006600)
> HEAD /myapp/status HTTP/2
> Host: dev.myapp.westus.cld.something.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 404 
HTTP/2 404 
< location: https://dev.myapp.westus.cld.something.com/myapp/status
location: https://dev.myapp.westus.cld.something.com /myapp/status
< date: Wed, 31 Jul 2019 19:10:13 GMT
date: Wed, 31 Jul 2019 19:10:13 GMT
< server: istio-envoy
server: istio-envoy

< 
* Connection #0 to host dev.myapp.westus.cld.something.com left intact

However when I try with

curl -Iv --insecure https://nonprod.westus.cloudapp.azure.com/myapp/status

I get HTTP 200. see below

*   Trying 123.12.123.12...
* TCP_NODELAY set
* Connected to nonprod.westus.cloudapp.azure.com (123.12.123.12) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: 

ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=****; L=*****; O=******, Inc.; CN=*********
*  start date: Jul 26 20:32:59 2019 GMT
*  expire date: Jul 26 20:32:59 2021 GMT
*  issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign RSA OV SSL CA 2018
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fb4b7003c00)
> HEAD /myapp/status HTTP/2
> Host: nonprod.westus.cloudapp.azure.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
HTTP/2 200 
< request-context: appId=cid-v1:39296fe8-1dc6-4049-bb31-e7816e83b9ec
request-context: appId=cid-v1:39296fe8-1dc6-4049-bb31-e7816e83b9ec
< access-control-allow-origin: *
access-control-allow-origin: *
< access-control-expose-headers: Content-Disposition
access-control-expose-headers: Content-Disposition
< content-type: text/plain;charset=ISO-8859-1
content-type: text/plain;charset=ISO-8859-1
< content-length: 69
content-length: 69
< date: Wed, 31 Jul 2019 19:07:54 GMT
date: Wed, 31 Jul 2019 19:07:54 GMT
< x-envoy-upstream-service-time: 7
x-envoy-upstream-service-time: 7
< server: istio-envoy
server: istio-envoy

< 
* Connection #0 to host nonprod.westus.cloudapp.azure.com left intact

Is my VS configuration wrong with regards to accepting HTTPS traffic? I am unable to look at envoy logs to see if there is any error.

How can I pinpoint where the issue is?

Update

dev.myapp.westus.cld.something.com is a CName record which ultimately points to nonprod.westus.cloudapp.azure.com

nonprod.westus.cloudapp.azure.com is a actual DNS which is mapped to a public IP.

Note sure if that makes any difference.

-- Shantanoo K
azure
azure-aks
istio
kubernetes

0 Answers