New user can view all pods without any rolebindings

4/19/2019

kube-apiserver.service is running with --authorization-mode=Node,RBAC

$ kubectl api-versions | grep rbac
rbac.authorization.k8s.io/v1
rbac.authorization.k8s.io/v1beta1

Believe this is enough to enable RBAC.

However, any new user i create can view all resources without any rolebindings.

Steps to create new user:

$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes nonadmin-csr.json | cfssljson -bare nonadmin
$ kubectl config set-cluster nonadmin --certificate-authority ca.pem --server https://127.0.0.1:6443
$ kubectl config set-credentials nonadmin --client-certificate nonadmin.pem --client-key nonadmin-key.pem
$ kubectl config set-context nonadmin --cluster nonadmin --user nonadmin
$ kubectl config use-context nonadmin

User nonadmin can view pods, svc without any rolebindings

$ kubectl get svc --all-namespaces
NAMESPACE       NAME                      TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE                 
default         kubernetes                ClusterIP   10.32.0.1     <none>        443/TCP                      5d4h
ingress-nginx   ingress-nginx             NodePort    10.32.0.129   <none>        80:30989/TCP,443:30686/TCP   5d3h
kube-system     calico-typha              ClusterIP   10.32.0.225   <none>        5473/TCP                     5d3h
kube-system     kube-dns                  ClusterIP   10.32.0.10    <none>        53/UDP,53/TCP                5d3h
rook-ceph       rook-ceph-mgr             ClusterIP   10.32.0.2     <none>        9283/TCP                     4d22h
rook-ceph       rook-ceph-mgr-dashboard   ClusterIP   10.32.0.156   <none>        8443/TCP                     4d22h
rook-ceph       rook-ceph-mon-a           ClusterIP   10.32.0.55    <none>        6790/TCP                     4d22h
rook-ceph       rook-ceph-mon-b           ClusterIP   10.32.0.187   <none>        6790/TCP                     4d22h
rook-ceph       rook-ceph-mon-c           ClusterIP   10.32.0.128   <none>        6790/TCP                     4d22h

Version:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:35:51Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:28:14Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}

This is an unmanaged kubernetes setup on Ubuntu 18 VMs. Where am i going wrong?

Edit1: Adding kubectl config view

$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /home/dadmin/ca.pem
    server: https://192.168.1.111:6443
  name: gabbar
- cluster:
    certificate-authority: /home/dadmin/ca.pem
    server: https://127.0.0.1:6443
  name: nonadmin
- cluster:
    certificate-authority: /home/dadmin/ca.pem
    server: https://192.168.1.111:6443
  name: kubernetes
contexts:
- context:
    cluster: gabbar
    namespace: testing
    user: gabbar
  name: gabbar
- context:
    cluster: nonadmin
    user: nonadmin
  name: nonadmin
- context:
    cluster: kubernetes
    user: admin
  name: kubernetes
current-context: nonadmin
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate: /home/dadmin/admin.pem
    client-key: /home/dadmin/admin-key.pem
- name: gabbar
  user:
    client-certificate: /home/dadmin/gabbar.pem
    client-key: /home/dadmin/gabbar-key.pem
- name: nonadmin
  user:
    client-certificate: /home/dadmin/nonadmin.pem
    client-key: /home/dadmin/nonadmin-key.pem

Edit 2: Solution as suggested by @VKR:

cat > operator-csr.json <<EOF
{
  "CN": "operator",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "IN",
      "L": "BGLR",
      "O": "system:view",  <==== HERE
      "OU": "CKA"
    }
  ]
}
EOF

cfssl gencert \
  -ca=ca.pem \
  -ca-key=ca-key.pem \
  -config=ca-config.json \
  -profile=kubernetes \
  operator-csr.json | cfssljson -bare operator

MasterNode~$ kubectl config set-cluster operator --certificate-authority ca.pem --server $SERVER
Cluster "operator" set.

MasterNode~$ kubectl config set-credentials operator --client-certificate operator.pem --client-key operator-key.pem
User "operator" set.

MasterNode~$ kubectl config set-context operator --cluster operator --user operator
Context "operator" created.

MasterNode~$ kubectl auth can-i get pods --as operator
no

MasterNode~$ kubectl create rolebinding operator --clusterrole view --user operator -n default --save-config
rolebinding.rbac.authorization.k8s.io/operator created

MasterNode~$ cat crb-view.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: view
subjects:
- kind: User
  name: operator
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: view
  apiGroup: rbac.authorization.k8s.io

MasterNode~$ kubectl create -f crb-view.yml --record --save-config
clusterrolebinding.rbac.authorization.k8s.io/view created

MasterNode~$ kubectl auth can-i get pods --as operator --all-namespaces
yes

MasterNode~$ kubectl auth can-i create pods --as operator --all-namespaces
no

MasterNode~$ kubectl config use-context operator
Switched to context "operator".

MasterNode~$ kubectl auth can-i "*" "*"
no

MasterNode~$ kubectl run db --image mongo
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
Error from server (Forbidden): deployments.apps is forbidden: User "operator" cannot create resource "deployments" in API group "apps" in the namespace "default"
-- papu
kubernetes

1 Answer

6/6/2019

Most probably root cause of such behavior is that use set "O": "system:masters" group while generating nonadmin-csr.json

system:masters group bounds to the cluster-admin super-user default role and as a result - every newly created user will have full access.

Here is a good article that provide you step-by-step instruction on how to create users with limited namespace access.

Quick test shows that similar users but with different groups have huge access differences

-subj "/CN=employee/O=testgroup" :

kubectl --context=employee-context get pods --all-namespaces
Error from server (Forbidden): pods is forbidden: User "employee" cannot list resource "pods" in API group "" at the cluster scope

-subj "/CN=newemployee/O=system:masters" :

kubectl --context=newemployee-context get pods --all-namespaces
NAMESPACE       NAME                                        READY   STATUS    RESTARTS   AGE
ingress-nginx   nginx-ingress-controller-797b884cbc-pckj6   1/1     Running   0          85d
ingress-nginx   prometheus-server-8658d8cdbb-92629          1/1     Running   0          36d
kube-system     coredns-86c58d9df4-gwk28                    1/1     Running   0          92d
kube-system     coredns-86c58d9df4-jxl84                    1/1     Running   0          92d
kube-system     etcd-kube-master-1                          1/1     Running   0          92d
kube-system     kube-apiserver-kube-master-1                1/1     Running   0          92d
kube-system     kube-controller-manager-kube-master-1       1/1     Running   4          92d
kube-system     kube-flannel-ds-amd64-k6sgd                 1/1     Running   0          92d
kube-system     kube-flannel-ds-amd64-mtrnc                 1/1     Running   0          92d
kube-system     kube-flannel-ds-amd64-zdzjl                 1/1     Running   1          92d
kube-system     kube-proxy-4pm27                            1/1     Running   1          92d
kube-system     kube-proxy-ghc7w                            1/1     Running   0          92d
kube-system     kube-proxy-wsq4h                            1/1     Running   0          92d
kube-system     kube-scheduler-kube-master-1                1/1     Running   4          92d
kube-system     tiller-deploy-5b7c66d59c-6wx89              1/1     Running   0          36d
-- VKR
Source: StackOverflow