GKE with Identity Aware Proxy returns Error code 9

2/17/2019

I have a dockerized flask application that running on kuberneetes in Google Cloud Platform with Identity-Aware Proxy enabled. I can run a "Hello World" website but when I try to use signed JWT headers then problems occur.

In my browser I am presented with

There was a problem with your request. Error code 9

My app is like this example and I use gunicorn to run the app. It seems that trouble happens in the first line

jwt = request.headers.get('x-goog-iap-jwt-assertion')

but that just makes no sense to me. But I can return a string before that line but not after. Any suggestions?

Details on the current kubernetes cluster

apiVersion: apps/v1
kind: Deployment
metadata:
  name: internal-tools-app
spec:
  selector:
    matchLabels:
      app: internal-tools
  template:
    metadata:
      labels:
        app: internal-tools
    spec:
      containers:
      - name: internal-web-app
        image: <<MY_IMAGE>>
---
apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
  name: internal-tools-backend-config
  namespace: default
spec:
  iap:
    enabled: true
    oauthclientCredentials:
      secretName: internal-tools-oauth
---
apiVersion: v1
kind: Service
metadata:
  name: internal-tools-service
  annotations:
    beta.cloud.google.com/backend-config: '{"default": "internal-tools-backend-config"}'
spec:
  type: NodePort
  selector:
    app: internal-tools
  ports:
  - name: it-first-port
    protocol: TCP
    port: 8080
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.global-static-ip-name: internal-tools-ip
    ingress.gcp.kubernetes.io/pre-shared-cert: "letsencrypt-internal-tools"
  name: internal-tools-ingress
spec:
  rules:
  - host: <<MY_DOMAIN>>
    http:
      paths:
      - backend:
          serviceName: internal-tools-service
          servicePort: it-first-port

EDIT

Further investigations show

ImportError: Error loading shared library libssl.so.45: No such file or directory (needed by /usr/local/lib/python3.6/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so)

when running the following

jwt.decode(
    iap_jwt, key,
    algorithms=['ES256'],
    audience=expected_audience)
-- mr.bjerre
google-cloud-platform
google-kubernetes-engine

2 Answers

10/1/2019

I just fixed this error code tonight by deleting and recreating my frontend and google-managed cert objects in GCP console. It seems to happen when I decommissioned and repurposed a cluster and deployed my app on a brand new cluster with same static IP address.

-- kot
Source: StackOverflow

5/7/2020

I got this answer from the Google Cloud Team bug tracker:

The Error code 9 occurs when multiple requests for re-authentication occur simultaneously (in particular, often caused by browsers reloading multiple windows/tabs at once). This flow currently requires for a temporary cookie flow to succeed first, and this cookie is unique to that flow. However if one flow starts before the previous one finishes, for example with multiple simultaneous refreshes in the same browser, this will cause the error you saw, and cause users to face that auth page.

You can try below options to overcome the issue

  • reboot 1 browser
  • clear cookies
  • better handling of sessions implementing

⁠session handlers

https://issuetracker.google.com/issues/155005454

-- Breedly
Source: StackOverflow