NGINX Ingress external oauth with Azure Active Directory

1/16/2020

I want to use Azure Active Directory as an external oauth2 provider to protect my services on the ingress level. In the past, I used basic ouath and everything worked like expected. But nginx provides the extern ouath methode which sounds much more confortable!

For that I created an SP:

$ az ad sp create-for-rbac  --skip-assignment --name test -o table

AppId                 DisplayName        Name               Password                   Tenant
<AZURE_CLIENT_ID>     test               http://test        <AZURE_CLIENT_SECRET>      <TENANT_ID>

My ingress ressource:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: ingress-nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/auth-url: "https://\$host/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://\$host/oauth2/start?rd=$escaped_request_uri"
    # nginx.ingress.kubernetes.io/auth-type: basic
    # nginx.ingress.kubernetes.io/auth-secret: basic-auth
    # nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'

And the externel-oauth:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: oauth2-proxy
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: oauth2-proxy
  template:
    metadata:
      labels:
        app: oauth2-proxy
    spec:
      containers:
      - args:
        - --provider=azure
        - --email-domain=microsoft.com
        - --upstream=file:///dev/null
        - --http-address=0.0.0.0:4180
        - --azure-tenant=$AZURE_TENANT_ID
        env:
          - name: OAUTH2_PROXY_CLIENT_ID
            value: $API_CLIENT_ID
          - name: OAUTH2_PROXY_CLIENT_SECRET
            value: $API_CLIENT_SECRET
          - name: OAUTH2_PROXY_COOKIE_SECRET
            value: $API_COOKIE_SECRET
          # created by docker run -ti --rm python:3-alpine python -c 'import secrets,base64; print(base64.b64encode(base64.b64encode(secrets.token_bytes(16))));
        image: docker.io/colemickens/oauth2_proxy:latest
        imagePullPolicy: Always
        name: oauth2-proxy
        ports:
        - containerPort: 4180
          protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: oauth2-proxy
  namespace: kube-system
spec:
  ports:
  - name: http
    port: 4180
    protocol: TCP
    targetPort: 4180
  selector:
    app: oauth2-proxy

It looks like something wents wrong but I have no idea what I missed. When i try to enter the page it loads up to one minute and ends in a '500 internal server error'. The logs of the ingress controller show an infinity loop of following:

10.244.2.1 - - [16/Jan/2020:15:32:30 +0000] "GET /oauth2/auth HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/xxxxx Safari/xxxx" 727 0.003 [upstream-default-backend] [] - - - - <AZURE_CLIENT_ID>
-- Nico Schuck
azure
azure-active-directory
kubernetes
nginx-ingress

1 Answer

1/16/2020

So you need another ingress for the oAuth deployment as well. here's how my setup looks like:

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: grafana-ingress-oauth
  namespace: grafana 
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
    - host: xxx
      http:
        paths:
          - path: /oauth2
            backend:
              serviceName: oauth2-proxy
              servicePort: 4180
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: grafana
  annotations:
    kubernetes.io/ingress.class: "nginx"
    kubernetes.io/tls-acme: "true"
    certmanager.k8s.io/cluster-issuer: letsencrypt-production
    ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
spec:
  rules:
    - host: xxx
      http:
        paths:
          - path: /
            backend:
              serviceName: grafana
              servicePort: 80

this way second ingress redirects to first and first does the auth and redirects back

-- 4c74356b41
Source: StackOverflow