K
Q

how to enable admission controller plugin on k8s where API server is deployed as a systemd service?

11/26/2019

I'm trying to apply podSecurityPolicy and try to test whether it's allowing me to create privileged pod. Below is the podSecurityPolicy resource manifest.

kind: PodSecurityPolicy
apiVersion: policy/v1beta1
metadata:
 name: podsecplcy
spec:
 hostIPC: false
 hostNetwork: false
 hostPID: false
 privileged: false
 readOnlyRootFilesystem: true
 hostPorts:
 - min: 10000
   max: 30000
 runAsUser:
  rule: RunAsAny
 fsGroup:
    rule: RunAsAny
 supplementalGroups:
    rule: RunAsAny
 seLinux:
    rule: RunAsAny
 volumes:
 - '*'

current psp as below

[root@master ~]# kubectl get psp
NAME         PRIV    CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
podsecplcy   false          RunAsAny   RunAsAny    RunAsAny   RunAsAny   true             *
[root@master ~]#

After submitted the above manifest,i'm trying to create privileged pod using below manifest.

apiVersion: v1
kind: Pod
metadata:
  name: pod-privileged
spec:
  containers:
  - name: main
    image: alpine
    command: ["/bin/sleep", "999999"]
    securityContext:
      privileged: true

Without any issues the pod is created.I hope it should throw error since privileged pod creation is restricted through podSecurityPolicy. Then i realized,it may be a admission controller plugin is not enabled and i saw which admission controller plugins are enabled by describe the kube-apiserver pod(Removed some lines for readability purpose) and able to see only NodeRestriction is enabled

[root@master ~]# kubectl -n kube-system describe po kube-apiserver-master.k8s
      --client-ca-file=/etc/kubernetes/pki/ca.crt
      --enable-admission-plugins=NodeRestriction
      --enable-bootstrap-token-auth=true

**Attempt:**
Tried to edit /etc/systemd/system/multi-user.target.wants/kubelet.service and changed ExecStart=/usr/bin/kubelet --enable-admission-plugins=Initializers,NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota
restarted kubelet service.But no luck

Now how to enable other admission controller plugins?
-- user10912187
kubernetes

Similar Questions

Istio Ingress Regex Negation
Minikube not pull image from local docker container registry
Prometheus PostgresSQL exporter without SSL and with password from Kubernetes secret
Job with Kubernetes Pod Operator fail while running with kubernetes Executor
MongoDB Unable to write process id 1 to file mongod.lock

1 Answer

11/26/2019

1. Locate the static pod manifest-path

From systemd status, you will be able to locate the kubelet unit file systemctl status kubelet.service

Do cat /etc/systemd/system/kubelet.service (replace path with the one you got from above command) Go to the directory which is pointing to --pod-manifest-path=

2. Open the yaml which starts kube-apiserver-master.k8s Pod

Example steps to locate YAML is below

cd /etc/kubernetes/manifests/
grep kube-apiserver-master.k8s *

3. Append PodSecurityPolicy to flag --enable-admission-plugins= in YAML file

4. Create a PSP and corresponding bindings for kube-system namespace

Create a PSP to grant access to pods in kube-system namespace including CNI

kubectl apply -f - <<EOF
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
 annotations:
  seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
 name: privileged
spec:
 allowedCapabilities:
 - '*'
 allowPrivilegeEscalation: true
 fsGroup:
  rule: 'RunAsAny'
 hostIPC: true
 hostNetwork: true
 hostPID: true
 hostPorts:
 - min: 0
   max: 65535
 privileged: true
 readOnlyRootFilesystem: false
 runAsUser:
  rule: 'RunAsAny'
 seLinux:
  rule: 'RunAsAny'
 supplementalGroups:
  rule: 'RunAsAny'
 volumes:
  - '*'
EOF

Cluster role which grants access to the privileged pod security policy

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 name: privileged-psp
rules:
 - apiGroups:
   - policy
   resourceNames:
   - privileged
   resources:
   - podsecuritypolicies
   verbs:
   - use
EOF

Role binding

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: kube-system-psp
 namespace: kube-system
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: privileged-psp
subjects:
 - apiGroup: rbac.authorization.k8s.io
   kind: Group
   name: system:nodes
 - apiGroup: rbac.authorization.k8s.io
   kind: Group
   name: system:serviceaccounts:kube-system
EOF
-- Ansil
Source: StackOverflow