I'm trying to deploy a container in a Kubernetes Kind cluster. The container I'm trying to deploy needs a couple of sysctls flags to be set.
The deployment fails with
forbidden sysctl: "kernel.msgmnb" not whitelisted
UPDATE
I have since added a cluster policy as suggested, created a role that grants usage to it and assigned the Cluster Role to the default service account:
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: sysctl-psp
spec:
privileged: false # Don't allow privileged pods!
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'
allowedUnsafeSysctls:
- kernel.msg*
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: role_allow_sysctl
rules:
- apiGroups: ['policy']
resources: ['podsecuritypolicies']
verbs: ['*']
resourceNames:
- sysctl-psp
- apiGroups: ['']
resources:
- replicasets
- services
- pods
verbs: ['*']
- apiGroups: ['apps']
resources:
- deployments
verbs: ['*']
The cluster role binding is like this:
kubectl -n <namespace> create rolebinding default:role_allow_sysctl --clusterrole=role_allow_sysctl --serviceaccount=<namespace>:default
I am then trying to create a deployment and a service in the same namespace:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-app
labels:
app: test-app
spec:
selector:
matchLabels:
app: test-app
tier: dev
strategy:
type: Recreate
template:
metadata:
labels:
app: test-app
tier: dev
spec:
securityContext:
sysctls:
- name: kernel.msgmnb
value: "6553600"
- name: kernel.msgmax
value: "1048800"
- name: kernel.msgmni
value: "32768"
- name: kernel.sem
value: "128 32768 128 4096"
containers:
- image: registry:5000/<container>:1.0.0
name: test-app
imagePullPolicy: IfNotPresent
ports:
- containerPort: 10666
name:port-1
---
The problem remains the same however, I'm getting multiple pods spawned, all failing with the same message forbidden sysctl: "kernel.msgmnb" not whitelisted
I don't think that --alowed-unsafe-sysctls
flag could work with Kind nodes, because Kind nodes themselves are containers, whose sysctl FS is read-only.
My workaround is to change the needed sysctl values on my host machine. Kind nodes (and in turn their containers) will reuse these values.
What needs to be done in your use case is Setting Sysctls for a Pod:
Use the pod securityContext to configure namespaced sysctls. The securityContext applies to all containers in the same pod.
In the documentation above you will find the necessary details and an example of how to configure a pod's securityContext.
Also remember that:
Warning: If you whitelist unsafe sysctls via the allowedUnsafeSysctls field in a PodSecurityPolicy, any pod using such a sysctl will fail to start if the sysctl is not whitelisted via the
--allowed-unsafe-sysctls
kubelet flag as well on that node.
Please let me know if that helps.