Whitelisting sysctls for containers in Kubernetes Kind

3/16/2020

I'm trying to deploy a container in a Kubernetes Kind cluster. The container I'm trying to deploy needs a couple of sysctls flags to be set.

The deployment fails with

forbidden sysctl: "kernel.msgmnb" not whitelisted

UPDATE

I have since added a cluster policy as suggested, created a role that grants usage to it and assigned the Cluster Role to the default service account:

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: sysctl-psp
spec:
  privileged: false  # Don't allow privileged pods!
  # The rest fills in some required fields.
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  volumes:
  - '*'
  allowedUnsafeSysctls:
  - kernel.msg*
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: role_allow_sysctl
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['*']
  resourceNames:
  - sysctl-psp
- apiGroups: ['']
  resources:
  - replicasets
  - services
  - pods
  verbs: ['*']
- apiGroups: ['apps']
  resources:
  - deployments
  verbs: ['*']

The cluster role binding is like this:

kubectl -n <namespace> create rolebinding default:role_allow_sysctl --clusterrole=role_allow_sysctl --serviceaccount=<namespace>:default

I am then trying to create a deployment and a service in the same namespace:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-app
  labels:
    app: test-app
spec:
  selector:
    matchLabels:
      app: test-app
      tier: dev
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: test-app
        tier: dev
    spec:
      securityContext:
        sysctls:
        - name: kernel.msgmnb
          value: "6553600"
        - name: kernel.msgmax
          value: "1048800"
        - name: kernel.msgmni
          value: "32768"
        - name: kernel.sem
          value: "128 32768 128 4096"
      containers:
      - image: registry:5000/<container>:1.0.0
        name: test-app
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 10666
          name:port-1
---

The problem remains the same however, I'm getting multiple pods spawned, all failing with the same message forbidden sysctl: "kernel.msgmnb" not whitelisted

-- Tudor Vintilescu
containers
kind
kubernetes
sysctl

2 Answers

4/12/2020

I don't think that --alowed-unsafe-sysctls flag could work with Kind nodes, because Kind nodes themselves are containers, whose sysctl FS is read-only.

My workaround is to change the needed sysctl values on my host machine. Kind nodes (and in turn their containers) will reuse these values.

-- Trần Việt Hoàng
Source: StackOverflow

3/17/2020

What needs to be done in your use case is Setting Sysctls for a Pod:

Use the pod securityContext to configure namespaced sysctls. The securityContext applies to all containers in the same pod.

In the documentation above you will find the necessary details and an example of how to configure a pod's securityContext.

Also remember that:

Warning: If you whitelist unsafe sysctls via the allowedUnsafeSysctls field in a PodSecurityPolicy, any pod using such a sysctl will fail to start if the sysctl is not whitelisted via the --allowed-unsafe-sysctls kubelet flag as well on that node.

Please let me know if that helps.

-- OhHiMark
Source: StackOverflow