I have a Daemonset running in privileged mode in a kubernetes cluster. This is the YAML spec of the daemon set.
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: my-daemon
spec:
template:
metadata:
labels:
app: my-daemon
spec:
hostNetwork: true
serviceAccountName: my-sa-account
containers:
- name: my-daemon
image: akhilerm/my-daemon:0.5
imagePullPolicy: Always
securityContext:
privileged: true
...
...Instead of using privileged:true, I am moving on to linux capabilties to give permissions to the DaemonSet. Therefore, I added all the linux capabilities to the container and removed privileged:true. This is the new YAML spec
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: my-daemon
spec:
template:
metadata:
labels:
app: my-daemon
spec:
hostNetwork: true
serviceAccountName: my-sa-account
containers:
- name: my-daemon
image: akhilerm/my-daemon:0.5
imagePullPolicy: Always
securityContext:
capabilities:
add: ["NET_BROADCAST", "NET_ADMIN", ..all CAPs..,"SYS_ADMIN"]
...
...
But when using with linux capabilities the daemon is not behaving as expected. In both cases the permission bitmap in /proc/1/status inside the container is same.
...
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000010000
SigIgn: 0000000000000004
SigCgt: 0000000000014002
CapInh: 0000003fffffffff
CapPrm: 0000003fffffffff
CapEff: 0000003fffffffff
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
...Is there any more fields or permissions that I need to set while using linux capabilities with a pod in kubernetes?
I am not 100% sure, but this issue may be in the missing capabilities: field.
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: my-daemon
spec:
template:
metadata:
labels:
app: my-daemon
spec:
hostNetwork: true
serviceAccountName: my-sa-account
containers:
- name: my-daemon
image: akhilerm/my-daemon:0.5
imagePullPolicy: Always
securityContext:
capabilities:
add: ["NET_BROADCAST", "NET_ADMIN", ..all CAPs..,"SYS_ADMIN"]
...
...
or
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: my-daemon
spec:
template:
metadata:
labels:
app: my-daemon
spec:
hostNetwork: true
serviceAccountName: my-sa-account
containers:
- name: my-daemon
image: akhilerm/my-daemon:0.5
imagePullPolicy: Always
securityContext:
capabilities:
add:
- NET_BROADCAST
- NET_ADMIN
- ...
- SYS_ADMIN
...
...Test it and let me know if there is any success. If not - I will try to dig deeper and provide you another solution. I found exactly this format in all the examples, so hope it will help. As a real example, I can provide you kubernetes-the-not-so-hard-way-with-ansible-ingress-with-traefik article with next explanations:
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
Without this setting we won’t be able to bind Traefik on port 80 and 443 (which is basically true for all services that want to use ports < 1024). We can also use privileged: true but this makes the attack surface way bigger. So using Linux Capabilities give us fine-grained control over superuser permissions and reduce the permissions to a minimum.
Or 1 more from Kubernetes official security-context article, but this time configuration is for the Pod:
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo-4
spec:
containers:
- name: sec-ctx-4
image: gcr.io/google-samples/node-hello:1.0
securityContext:
capabilities:
add: ["NET_ADMIN", "SYS_TIME"]Hope this will help you.