Learning k8s + istio here. I've setup a 2 nodes + 1 master cluster with kops. I have Istio as ingress controller. I'm trying to set up OIDC Auth for a dummy nginx service. I'm hitting a super weird bug I have no idea where it's coming from.

So, I have a

1. Keycloak service
2. Nginx service

The keycloak service runs on keycloak.example.com The nginx service runs on example.com

There is a Classic ELB on AWS to serve that. There are Route53 DNS records for

ALIAS example.com          dualstack.awdoijawdij.amazonaws.com
ALIAS keycloak.example.com dualstack.awdoijawdij.amazonaws.com

When I was setting up the keycloak service, and there was only that service, I had no problem. But when I added the dummy nginx service, I started getting this.

I would use firefox to go to keycloak.example.com, and get a 404. If I do a hard refresh, then the page loads.

Then I would go to example.com, and would get a 404. If I do a hard refresh, then the page loads.

If I do a hard refresh on one page, then when I go to the other page, I will have to do a hard reload or I get a 404. It's like some DNS entry is toggling between these two things whenever I do the hard refresh. I have no idea on how to debug this.

If I

• wget -O- example.com I have a 301 redirect to https://example.com as expected
• wget -O- https://example.com I have a 200 OK as expected
• wget -O- keycloak.example.com I have a 301 redirect to https://keycloak.example.com as expected
• wget -O- https://keycloak.example.com I have a 200 OK as expected

Then everything is fine. Seems like the problem only occurs in the browser.

I tried opening the pages in Incognito mode, but the problem persists.

Can someone help me in debugging this?

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
    - port: 80
      name: http
      protocol: TCP
  selector:
    app: nginx
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: nginx-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      tls:
        httpsRedirect: true
      hosts:
        - "example.com"
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: ingress-cert
      hosts:
        - "example.com"
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
    - "example.com"
  gateways:
    - nginx-gateway
  http:
    - route:
        - destination:
            port:
              number: 80
            host: nginx
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: keycloak-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      tls:
        httpsRedirect: true
      hosts:
        - "keycloak.example.com"
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: ingress-cert
      hosts:
        - "keycloak.example.com"
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: keycloak
spec:
  hosts:
    - "keycloak.example.com"
  gateways:
    - keycloak-gateway
  http:
    - route:
        - destination:
            port:
              number: 80
            host: keycloak-http