I am seeing an RBAC failure when trying to deploy KubeIP to GKE.
I have isolated the issue down to the following section of the KubeIP infrastructure:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubeip-sa
namespace: kube-system
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get","list","watch","patch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get","list","watch"]
I am getting the following error from kubectl
and GKE:
Error from server (Forbidden): error when creating "template.yml": clusterroles.rbac.authorization.k8s.io "kubeip-sa" is forbidden: attempt to grant extra privileges: [{[get] [] [nodes] [] []} {[list] [] [nodes] [] []} {[watch] [] [nodes] [] []} {[patch] [] [nodes] [] []} {[get] [] [pods] [] []} {[list] [] [pods] [] []} {[watch] [] [pods] [] []}] user=&{108986779198363313539 [system:authenticated] map[user-assertion.cloud.google.com:[AKUJVpldMDXqrDZ2slnJReDbLytxt6P2EEyEBbLNRB90oOATH4vIURo/lIhaBuAj9nnwwyxJDSxj2OdCyjjgBC/s5QxftIJnr8128ToTglCzk+e8Wybt4heIizRHugWnIhKNqkF+B0yiv0pIxgOfakma+SbkzbQbVzJPtgxsmHmak30YfPA58n/xyJ8R7oNVJ5dFUAWDFNsqHf/auolViw0Zd7Cr4aYYDXX4GScw==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
I have crated the appropriate ~/.kube/config by issuing
gcloud container clusters get-credentials <cluster> \
--zone <zone> \
--project <project>
The gcloud service account I am using has been granted cluster-admin in the GKE cluster in question
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud config get-value account)
I can verify that my service account user should have the cluster-admin role checking my current gcloud user and checking the GKE ClusterRoleBinding
$ gcloud config get-value account
terraform@<project>.iam.gserviceaccount.com
$ kubectl describe clusterrolebinding cluster-admin-binding
Name: cluster-admin-binding
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
User terraform@<project>.iam.gserviceaccount.com
According to kubectl I should be able to create ClusterRoleBindings
$ kubectl auth can-i create clusterrolebinding
yes
Does anybody see what element of GKE RBAC I am missing?
The answer at this question "Creating a ClusterRole as the default compute service account fails with extra privileges error" guided me to the solution.
If you map the ClusterRoleBinding to the service account id instead of the email everything works as expected.
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud iam service-accounts describe <service account email> --format="value(uniqueId)")