I am try to using airflow kubernetes pod operator, and I need to pass some security context when creating the pod in order to have privilege to mount s3fs, however it has no effect when I pass it with format below, what else could I do?
the security context I am try to pass is:
security_context = {
"privileged": True, # I have tried to pass "true", not working too
"capabilities": {
"add": ["SYS_ADMIN"]
}
}
and the corresponding config in yaml is:
securityContext:
privileged: true
capabilities:
add:
- SYS_ADMIN
the test code is:
#!/usr/bin/python
# -*- coding: utf-8 -*-
import datetime
from unittest import TestCase
from airflow.operators.kubernetes_pod_operator import KubernetesPodOperator
class TestKubernetesPodOperator(TestCase):
def setUp(self):
self.namespace = "airflow-test"
self.image = "airflow-dc/airflow-dc:v6.12.34"
self.name = "test"
self.config_file = "/home/think/.kube/config.yml"
self.cluster_context = "default"
self.dag_id = "test_onoff"
self.task_id = "root_test_onoff"
self.execution_date = datetime.datetime.now()
self.context = {"dag_id": self.dag_id,
"task_id": self.task_id,
"execution_date": self.execution_date}
self.cmds = ["echo"]
self.arguments = ["hello world"]
self.resources = {
"limits":
{"memory": "512Mi",
"cpu": "500m"},
"requests": {
"memory": "512Mi",
"cpu": "500m"}
}
# this is the security context that I passed to kubernetes, but it does not work
self.security_context = {
"privileged": True, # I have tried to pass "true", not working too
"capabilities": {
"add": ["SYS_ADMIN"]
}
}
self.operator = KubernetesPodOperator(
namespace=self.namespace, image=self.image, name=self.name,
cluster_context=self.cluster_context,
config_file=self.config_file,
cmds=self.cmds,
arguments=self.arguments,
startup_timeout_seconds=600,
is_delete_operator_pod=True,
resources=self.resources,
security_context=self.security_context,
**self.context)
def test_execute(self):
self.operator.execute(self.context)
I could start the pod with priority with below yaml with kubectl, but if I start pod with python, I just could not start it with privileged, and this is import for me, because I need to mount s3fs to save some data.
kind: Pod
apiVersion: v1
metadata:
name: test-pod-3
spec:
containers:
- name: test-pod-3
image: airflow-dc/airflow-dc:v6.12.34
command:
- "/bin/sh"
args:
- "-c"
- "touch /mnt/SUCCESS && sleep 60000 || exit 1"
resources:
limits:
memory: 512Mi
cpu: 500m
requests:
memory: 512Mi
cpu: 500m
securityContext:
privileged: true
capabilities:
add:
- SYS_ADMIN