I have tried now so many times setting up this pipeline in Azure devops where I want to deploy a AKS cluster and place istio on top.
Deploying the AKS using Terraform works great.
After this I try to install istio using helm but the command I use gives forbidden error.
helm.exe install --namespace istio-system --name istio-init --wait C:\Istio\install\kubernetes\helm\istio
I used the local path since this was the only good way I could find for helm to find the istio chart i have on the build agent.
The error message
Error: release istio-init failed: clusterroles.rbac.authorization.k8s.io "istio-galley-istio-system" is forbidden: attempt to grant extra privileges: [{[*] [admissionregistration.k8s.io] [validatingwebhookconfigurations] [] []} {[get] [config.istio.io] [*] [] []} {[list] [config.istio.io] [*] [] []} {[watch] [config.istio.io] [*] [] []} {[get] [*] [deployments] [istio-galley] []} {[get] [*] [endpoints] [istio-galley] []}] user=&{system:serviceaccount:kube-system:tillerserviceaccount 56632fa4-55e7-11e9-a4a1-9af49f3bf03a [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]} ownerrules=[] ruleResolutionErrors=[[clusterroles.rbac.authorization.k8s.io "cluster-admin" not found, clusterroles.rbac.authorization.k8s.io "system:discovery" not found, clusterroles.rbac.authorization.k8s.io "cluster-admin" not found, clusterroles.rbac.authorization.k8s.io "system:discovery" not found, clusterroles.rbac.authorization.k8s.io "system:discovery" not found, clusterroles.rbac.authorization.k8s.io "cluster-admin" not found]]
The serviceaccount I use (system:serviceaccount:kube-system:tillerserviceaccount as you can see in error message) are configured using this rbac config:
apiVersion: v1
kind: ServiceAccount
metadata:
name: tillerserviceaccount
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tillerbinding
roleRef:
apiGroup: ""
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tillerserviceaccount
namespace: kube-system
Still the error message says in the ruleResolutionErrors that it looks for cluster-admin but it is not found.
I even tried the extreme and set all service accounts as cluster admins to test:
kubectl create clusterrolebinding serviceaccounts-admins --clusterrole=cluster-admin --group=system:serviceaccounts
But even after that I get the same error with the same ruleResolutionErrors.
I am stuck and appriciate any help in what I can do differently.
this is the role binding we are using in dev clusters:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tillerbinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tillerserviceaccount
namespace: kube-system
edit: in this case the error was due to AKS created without RBAC.