I have an application that uses GKE Ingress (master version 1.10.6-gke.2) for a load balancer. Recently GKE started supporting declaring IAP support via BackendConfig
. I followed the documentation at [1] and [2]. However, now, GKE seems to hang while creating my Ingress.
Below is the yaml for my service, ingress and backendconfig.
kubectl -n randall-test-1 get svc,ing,backendconfig -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
annotations:
beta.cloud.google.com/backend-config: '{"default": "airflow-backend-config"}'
service.alpha.kubernetes.io/app-protocols: '{"web":"HTTPS"}'
creationTimestamp: 2018-09-10T19:23:13Z
name: airflow
namespace: randall-test-1
resourceVersion: "2155724"
selfLink: /api/v1/namespaces/randall-test-1/services/airflow
uid: X-X-X-X-X
spec:
clusterIP: X.X.X.X
externalTrafficPolicy: Cluster
ports:
- name: web
nodePort: 30099
port: 8080
protocol: TCP
targetPort: web
selector:
app: airflow
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
- apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.allow-http: "false"
creationTimestamp: 2018-09-10T19:23:13Z
generation: 1
name: airflow
namespace: randall-test-1
resourceVersion: "2155721"
selfLink: /apis/extensions/v1beta1/namespaces/randall-test-1/ingresses/airflow
uid: X-X-X-X-X
spec:
backend:
serviceName: airflow
servicePort: 8080
tls:
- secretName: tls
status:
loadBalancer: {}
- apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
clusterName: ""
creationTimestamp: 2018-09-10T19:23:13Z
generation: 1
name: airflow-backend-config
namespace: randall-test-1
resourceVersion: "2155728"
selfLink: /apis/cloud.google.com/v1beta1/namespaces/randall-test-1/backendconfigs/airflow-backend-config
uid: X-X-X-X-X
spec:
iap:
enabled: true
oauthclientCredentials:
secretName: oauth2
kind: List
metadata:
resourceVersion: ""
selfLink: ""
The hang gives me no insight.
cluster@master0:~/kube-config$ kubectl -n randall-test-1 describe ing
Name: airflow
Namespace: randall-test-1
Address:
Default backend: airflow:8080 (X.X.X.X:8080)
TLS:
tls terminates
Rules:
Host Path Backends
---- ---- --------
* * airflow:8080 (X.X.X.X:8080)
Annotations:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 6m loadbalancer-controller randall-test-1/airflow
However, in GKE console, I just get Creating ingress
as a status for > 20 mins with no resolution. I also check my Load Balancers
in console and see nothing.
Any ideas what is happening or what else I can check?
I also tried to do this with just securityPolicy
which is supposed to link the Load Balancer with a Cloud Armor policy. This also doesn't work with a similar hang.
[1] https://cloud.google.com/iap/docs/enabling-kubernetes-howto
[2] https://cloud.google.com/kubernetes-engine/docs/concepts/backendconfig
NOTE: cross posted at https://github.com/kubernetes/ingress-gce/issues/469
We've been getting a few similar cases in the past few days. There seems to be something wrong with permissions of the default GKE service account.
Can you try to add the following permissions to it: