kube-apiserver.service
is running with --authorization-mode=Node,RBAC
$ kubectl api-versions | grep rbac
rbac.authorization.k8s.io/v1
rbac.authorization.k8s.io/v1beta1
Believe this is enough to enable RBAC.
However, any new user i create can view all resources without any rolebindings.
Steps to create new user:
$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes nonadmin-csr.json | cfssljson -bare nonadmin
$ kubectl config set-cluster nonadmin --certificate-authority ca.pem --server https://127.0.0.1:6443
$ kubectl config set-credentials nonadmin --client-certificate nonadmin.pem --client-key nonadmin-key.pem
$ kubectl config set-context nonadmin --cluster nonadmin --user nonadmin
$ kubectl config use-context nonadmin
User nonadmin
can view pods, svc without any rolebindings
$ kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.32.0.1 <none> 443/TCP 5d4h
ingress-nginx ingress-nginx NodePort 10.32.0.129 <none> 80:30989/TCP,443:30686/TCP 5d3h
kube-system calico-typha ClusterIP 10.32.0.225 <none> 5473/TCP 5d3h
kube-system kube-dns ClusterIP 10.32.0.10 <none> 53/UDP,53/TCP 5d3h
rook-ceph rook-ceph-mgr ClusterIP 10.32.0.2 <none> 9283/TCP 4d22h
rook-ceph rook-ceph-mgr-dashboard ClusterIP 10.32.0.156 <none> 8443/TCP 4d22h
rook-ceph rook-ceph-mon-a ClusterIP 10.32.0.55 <none> 6790/TCP 4d22h
rook-ceph rook-ceph-mon-b ClusterIP 10.32.0.187 <none> 6790/TCP 4d22h
rook-ceph rook-ceph-mon-c ClusterIP 10.32.0.128 <none> 6790/TCP 4d22h
Version:
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:35:51Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:28:14Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
This is an unmanaged kubernetes setup on Ubuntu 18 VMs. Where am i going wrong?
Edit1: Adding kubectl config view
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/dadmin/ca.pem
server: https://192.168.1.111:6443
name: gabbar
- cluster:
certificate-authority: /home/dadmin/ca.pem
server: https://127.0.0.1:6443
name: nonadmin
- cluster:
certificate-authority: /home/dadmin/ca.pem
server: https://192.168.1.111:6443
name: kubernetes
contexts:
- context:
cluster: gabbar
namespace: testing
user: gabbar
name: gabbar
- context:
cluster: nonadmin
user: nonadmin
name: nonadmin
- context:
cluster: kubernetes
user: admin
name: kubernetes
current-context: nonadmin
kind: Config
preferences: {}
users:
- name: admin
user:
client-certificate: /home/dadmin/admin.pem
client-key: /home/dadmin/admin-key.pem
- name: gabbar
user:
client-certificate: /home/dadmin/gabbar.pem
client-key: /home/dadmin/gabbar-key.pem
- name: nonadmin
user:
client-certificate: /home/dadmin/nonadmin.pem
client-key: /home/dadmin/nonadmin-key.pem
Edit 2: Solution as suggested by @VKR:
cat > operator-csr.json <<EOF
{
"CN": "operator",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "IN",
"L": "BGLR",
"O": "system:view", <==== HERE
"OU": "CKA"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
operator-csr.json | cfssljson -bare operator
MasterNode~$ kubectl config set-cluster operator --certificate-authority ca.pem --server $SERVER
Cluster "operator" set.
MasterNode~$ kubectl config set-credentials operator --client-certificate operator.pem --client-key operator-key.pem
User "operator" set.
MasterNode~$ kubectl config set-context operator --cluster operator --user operator
Context "operator" created.
MasterNode~$ kubectl auth can-i get pods --as operator
no
MasterNode~$ kubectl create rolebinding operator --clusterrole view --user operator -n default --save-config
rolebinding.rbac.authorization.k8s.io/operator created
MasterNode~$ cat crb-view.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: view
subjects:
- kind: User
name: operator
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: view
apiGroup: rbac.authorization.k8s.io
MasterNode~$ kubectl create -f crb-view.yml --record --save-config
clusterrolebinding.rbac.authorization.k8s.io/view created
MasterNode~$ kubectl auth can-i get pods --as operator --all-namespaces
yes
MasterNode~$ kubectl auth can-i create pods --as operator --all-namespaces
no
MasterNode~$ kubectl config use-context operator
Switched to context "operator".
MasterNode~$ kubectl auth can-i "*" "*"
no
MasterNode~$ kubectl run db --image mongo
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
Error from server (Forbidden): deployments.apps is forbidden: User "operator" cannot create resource "deployments" in API group "apps" in the namespace "default"
Most probably root cause of such behavior is that use set "O": "system:masters"
group while generating nonadmin-csr.json
system:masters
group bounds to the cluster-admin super-user default role and as a result - every newly created user will have full access.
Here is a good article that provide you step-by-step instruction on how to create users with limited namespace access.
Quick test shows that similar users but with different groups have huge access differences
-subj "/CN=employee/O=testgroup" :
kubectl --context=employee-context get pods --all-namespaces
Error from server (Forbidden): pods is forbidden: User "employee" cannot list resource "pods" in API group "" at the cluster scope
-subj "/CN=newemployee/O=system:masters" :
kubectl --context=newemployee-context get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx nginx-ingress-controller-797b884cbc-pckj6 1/1 Running 0 85d
ingress-nginx prometheus-server-8658d8cdbb-92629 1/1 Running 0 36d
kube-system coredns-86c58d9df4-gwk28 1/1 Running 0 92d
kube-system coredns-86c58d9df4-jxl84 1/1 Running 0 92d
kube-system etcd-kube-master-1 1/1 Running 0 92d
kube-system kube-apiserver-kube-master-1 1/1 Running 0 92d
kube-system kube-controller-manager-kube-master-1 1/1 Running 4 92d
kube-system kube-flannel-ds-amd64-k6sgd 1/1 Running 0 92d
kube-system kube-flannel-ds-amd64-mtrnc 1/1 Running 0 92d
kube-system kube-flannel-ds-amd64-zdzjl 1/1 Running 1 92d
kube-system kube-proxy-4pm27 1/1 Running 1 92d
kube-system kube-proxy-ghc7w 1/1 Running 0 92d
kube-system kube-proxy-wsq4h 1/1 Running 0 92d
kube-system kube-scheduler-kube-master-1 1/1 Running 4 92d
kube-system tiller-deploy-5b7c66d59c-6wx89 1/1 Running 0 36d