Why Won't This Pod Validate Against This PSP?

11/25/2019

On our cluster, we use the following PSP and ClusterRoleBinding for almost all our services:

apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  allowPrivilegeEscalation: false
  fsGroup:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  requiredDropCapabilities:
  - ALL
  runAsUser:
    rule: MustRunAsNonRoot
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  volumes:
  - configMap
  - emptyDir
  - projected
  - secret
  - downwardAPI
  - persistentVolumeClaim
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: psp:restricted
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:restricted
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:authenticated
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: psp:restricted
rules:
- apiGroups:
  - extensions
  resourceNames:
  - restricted
  resources:
  - podsecuritypolicies
  verbs:
  - use

In other words, a restrictive PSP that doesn't allow anything exciting. This works well for us and is a well-understood and mature configuration. However, when creating pods with the Airflow GKEPodOperator, we get the following error:

Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "e1ab15d888c52b90c2fea73fec3a75cd3d79125f21f7840e2e6ca3c00f3b3b16" network for pod "braintree-to-gcs-11bd354a": NetworkPlugin cni failed to set up pod "braintree-to-gcs-11bd354a_private" network: connection is unauthorized: pods "braintree-to-gcs-11bd354a" is forbidden: unable to validate against any pod security policy: []

Unless I badly miss my guess, our Pod specification is failing to validate against this PSP. However, I'm stuck for why this might be the case - it's essentially the same kind of container we routinely run in production without any issues at all. My only guess is that we haven't set the container securityContext (as Airflow doesn't permit this) but I don't think this is actually necessary for the Pod to validate against the PSP. The Pod specification created by Airflow is as follows:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    # note that the pod is attempting to validate against a different PSP (but still failing)
    kubernetes.io/psp: dynatrace-oneagent
  name: braintree-to-gcs-11bd354a
  namespace: private
spec:
  affinity: {}
  containers:
  - args:
    - "2011-02-19T00:00:00+00:00"
    env:
     REDACTED
    image: gcr.io/prod-233014/c-braintree-etl:6ac00fdba4cf75ac3b13002d5d343f5427bbd44b
    imagePullPolicy: IfNotPresent
    name: base
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-pqmf6
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  imagePullSecrets:
  - name: gcr-json-key
  nodeName: gke-main-cluster-preemptible-nodes-1-e414988e-dn0d
  priority: 0
  restartPolicy: Never
  schedulerName: default-scheduler
  securityContext:
    runAsUser: 65532
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: default-token-pqmf6
    secret:
      defaultMode: 420
      secretName: default-token-pqmf6
-- Sam Gardner
kubernetes

0 Answers