I have tried to install an azure build agent on our on-prem kubernetes cluster. When trying to run builds, we always hit Status 403 Forbidden when executing kubectl
commands.
Since RBAC is enabled, I have setup a ServiceAccount, CluterRole and ClusterRoleBinding for our build agent namespace with the following rules:
kind: ServiceAccount
apiVersion: v1
metadata:
name: build-agent
namespace: agent_ns
automountServiceAccountToken: false
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: build-agent
rules:
- apiGroups:
- ""
resources:
- namespaces
- nodes
- nodes/proxy
- proxy
- pods
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: build-agent
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: build-agent
subjects:
- kind: ServiceAccount
name: build-agent
namespace: agent_ns
Still, if I sh into the pod using:
kubectl exec -it build-agent-xxxxxxxx-xxxx --namespace=agent_ns -- sh
And try to access the kubernetes api via curl I get 403:
curl https://10.x.x.x:6443
curl: (56) Received HTTP code 403 from proxy after CONNECT
The kubeconfig file is present in the pod VM. It is the same I use from the host machine when I kubectl to the cluster. Same as the curl command, if I try to run a kubectl command after sh-ing into the pod, the requests returns 403:
> kubectl get ns
Unable to connect to the server: Forbidden
EDIT:
Deployment description now also contains a service accout specification. Problem still persists:
> kubectl get pod build-agent-xxxxxxxxx-xxxxx -o yaml -n agent_ns
...
spec:
automountServiceAccountToken: false
...
serviceAccount: build-agent
serviceAccountName: build-agent
Am I missing something? I can provide more information if necessary, just let me know.
The cluster is running fine, we already have tiller, traefik and a service which is up and running.