K
Q

Pod Security Policy not working as intended

3/8/2019

I am creating a Pod Security Policy to stop ALL users from creating a Pod as Root user. My cluster is on GKE. The steps Ive carried out till now are

1) Enable PodSecurityPolicy in my cluster

gcloud beta container clusters update standard-cluster-11  --enable-pod-security-policy

2) Define the Policy. The policy is simple and restricts root user.

apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
metadata:
  name: a-restrict-root
spec:
  privileged: false
  runAsUser:
    rule: MustRunAsNonRoot  # <------ Root user restricted.
  seLinux:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - '*'

3) And then ofcourse implementing the correct RBAC rules so that it can be implemented for ALL users.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gce:podsecuritypolicy:a-restrict-root
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/cluster-service: "true"
rules:
- apiGroups:
  - policy
  resourceNames:
  - a-restrict-root
  resources:
  - podsecuritypolicies
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: gce:podsecuritypolicy:a-restrict-root
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/cluster-service: "true"
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: gce:podsecuritypolicy:a-restrict-root
subjects:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts

Now comes the part where I try to spin up a Pod. The pod definition looks like this :

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    runAsUser: 0
    fsGroup: 0
  volumes:
  - name: sec-ctx-vol
    emptyDir: {}
  containers:
  - name: sec-ctx-demo
    image: gcr.io/google-samples/node-hello:1.0
    volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/demo
    securityContext:
      allowPrivilegeEscalation: false

As you can see, the runAsUser is set to 0 meaning root.

When I run kubectl create -f pod.yaml, the pod is creating and goes into Running state.

When I exec into the Pod, I can see all process running as root

$ kubectl exec -it security-context-demo -- sh
# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.0  0.0   4336   812 ?        Ss   19:25   0:00 /bin/sh -c node server.js
root           6  0.4  0.5 772124 22656 ?        Sl   19:25   0:00 node server.js
root          11  0.0  0.0   4336   724 ?        Ss   19:26   0:00 sh
root          16  0.0  0.0  17500  2072 ?        R+   19:26   0:00 ps aux

But based on my PodSecurityPolicy this should NOT be allowed. Is there something I have missed ?

UPDATE :

I spun up a default nginx pod that I know always starts as root user. Its manifest is :

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx

And when I create it, it too starts up successfully.

$ kubectl get po
NAME                    READY     STATUS    RESTARTS   AGE
nginx                   1/1       Running   0          2m

Whereas because of the PSP, it should not start up.

--
google-kubernetes-engine
kubernetes

Similar Questions

Kubernetes - Unable to access apache Pod from proxy
Connect AKS Cluster through IPSEC VPN with only 1 private IP
Kubectl insufficient resource allocation in aws cluster
How to ensure only one pod runs on my node in GKE?
Azure Kubernetes Error when running &quot;az aks get-credentials --resource-group AKSResourceGroupName --name AKSClusterName&quot; command

2 Answers

3/8/2019

I got what you are missing just add this in your pod config file:

Before that, make sure to create the user (say, appuser) uid -> say, 999 and group (say, appgroup) gid ->say, 999 in the Docker container, and then try starting the container with that user and add:

securityContext:
  runAsUser: 999

This might be a good read: SecurityContext

Also, when you are doing this:

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    runAsUser: 0
    fsGroup: 0

You are overriding the PodSecurityPolicy see here

Update: 1

How to fix this:

apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
metadata:
  name: a-restrict-root
spec:
  privileged: false
  defautlAllowPrivilegeEscalation: false
  # This is redundant with non-root + disallow privilege escalation,
  # but we can provide it for defense in depth.
  requiredDropCapabilities:
    - ALL
  hostIPC: false
  hostPID: false
  runAsUser:
    # Require the container to run without root privileges.
    rule: 'MustRunAsNonRoot'
  seLinux:
    # This policy assumes the nodes are using AppArmor rather than SELinux.
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 1
        max: 65535
-- vancleff
Source: StackOverflow
3/8/2019

If you fetch the created pod from the API, it will contain an annotation indicating which PSP allowed the pod. I expect a different PSP is in place which allows the pod

-- Jordan Liggitt
Source: StackOverflow