Is it safe to delete the cluster-admin ClusterRoleBinding on open-source Kubernetes v1.14?

7/22/2020

I have a Kubernetes v1.14 on-prem cluster deployed using kubeadm and my kubernetes-admin user login has been compromised.

I want to revoke its certificate but the Kubernetes API server does not have a mechanism to lookup a CRL file. I already created another admin user using a new ClusterRoleBinding. The only solution I see is to de-authorize the user by removing it's associated ClusterRoleBinding: cluster-admin.

However, this ClusterRoleBinding's Subject is the "system:masters" group. Even though I couldn't find any other system user that belongs to the "system:masters" group I'm afraid it will break some things if I delete it. I don't have any experience with this kind of situation.

I googled a lot but I couldn't find relevant information on this topic for open-source Kubernetes.

Is it safe to delete the "cluster-admin" ClusterRoleBinding?

% openssl x509 -noout  -subject -in  <(kubectl config view --raw  -o jsonpath='{.users[?(@.name == "kubernetes-admin")].user.client-certificate-data}'  | base64 -d)
subject= /O=system:masters/CN=kubernetes-admin
% 

% kubectl describe clusterrolebindings.rbac.authorization.k8s.io cluster-admin 
Name:         cluster-admin
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
Role:
  Kind:  ClusterRole
  Name:  cluster-admin
Subjects:
  Kind   Name            Namespace
  ----   ----            ---------
  Group  system:masters  
% 
-- Iulian Mandache
kubernetes

1 Answer

7/23/2020

It's not safe to delete because system:masters is used as Subject in the client certificate used by Kubernetes API Server to interact with ETCD and kubelet.

To deal with the breach I would suggest below approach

  1. Take backup of ETCD
  2. Regenerate all certificates
  3. Restart all control plane components with newly generated certificates
  4. If any issue use the ETCD backup to restore.
-- Arghya Sadhu
Source: StackOverflow