Bug description
We have setup an istio over on eks cluster & a java app is hosted in it. The pod has been created along with service with type ClusterIP
We have created Virtual Service, Gateway & set the istio ingress gateway as a NodePort. In front of the istio ingress gateway, we placed the AWS Application Load Balancer. We created a route53(DNS) entry which points to the above said ALB
Now, all the services & pods are UP
Now, on hitting the DNS with https, the request info we get in the above said Spring Boot Java Application, has been changed from https to http with port 443
For Eg: if I make a curl request to say curl https://sarath.somedomain.com/helloworld
, in our java app, while we get the request info we get them as http://sarath.somedomain.com:443/helloworld
Also we have printed the headers like x-forwarded-proto, x-forwarded-port in our Java App.
x-forwarded-proto
value we get in the java app is http
x-forwarded-port
value we get in the java app is 443
x-forwarded-for
value we get in the java app is null
Our key suspicion is with envoy which does this protocol conversion from https to http
Expected behavior
If I hit say curl https://sarath.somedomain.com/helloworld
, in our java app we should get the request info as https://sarath.somedomain.com/helloworld
Steps to reproduce the bug
Gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sarath-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "sarath.somedomain.com"
- "prasath.somedomain.com"
Virtual Service:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: sp-vs
spec:
hosts:
- "sarath.somedomain.com"
gateways:
- sarath-gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
port:
number: 80
host: javaapp-svc
ALB Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: sarathingress
namespace: istio-system
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/subnets: subnet-XXXXXXXX,subnet-XXXXXXXX
alb.ingress.kubernetes.io/security-groups: sg-XXXXXXXX
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:XXXXXXXX:certificate/XXXXXXXX
spec:
rules:
- host: sarath.somedomain.com
http:
paths:
- path: /*
backend:
serviceName: istio-ingressgateway
servicePort: 80
App Service:
apiVersion: v1
kind: Service
metadata:
name: javaapp-svc
labels:
app: javaapp-svc
name: javaapp-svc
spec:
selector:
app: javaapp-svc
ports:
- name: http
port: 80
targetPort: 9090
protocol: TCP
Version (include the output of istioctl version --remote
and kubectl version
)
Output of istioctl version --remote
:
client version: 1.2.5 citadel version: 1.2.5 galley version: 1.2.5 ingressgateway version: 1.2.5 pilot version: 1.2.5 policy version: 1.2.5 sidecar-injector version: 1.2.5 telemetry version: 1.2.5
Output of kubectl version
:
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.7", GitCommit:"6f482974b76db3f1e0f5d24605a9d1d38fad9a2b", GitTreeState:"clean", BuildDate:"2019-03-29T16:15:10Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.10-eks-5ac0f1", GitCommit:"5ac0f1d9ab2c254ea2b0ce3534fd72932094c6e1", GitTreeState:"clean", BuildDate:"2019-08-20T22:39:46Z", GoVersion:"go1.11.13", Compiler:"gc", Platform:"linux/amd64"}
How was Istio installed?
Istio is installed through helm, below is the command used:
helm install install/kubernetes/helm/istio-init --name istio-init --namespace istio-system
helm template \
--set gateways.istio-ingressgateway.type=NodePort \
--set prometheus.enabled=true \
--set grafana.enabled=true \
--set tracing.enabled=true \
--set kiali.enabled=true \
--set "kiali.dashboard.jaegerURL=http://jaeger-query:16686" \
--set "kiali.dashboard.grafanaURL=http://grafana:3000" \
install/kubernetes/helm/istio \
--name istio --namespace istio-system > $HOME/istio.yaml
Environment where bug was observed (cloud vendor, OS, etc)
AWS, EC2 Machine, Amazon Linux v2