KubernetesPodOperator privileged security_context in Airflow

11/22/2019

I am running Airflow on Google's Cloud Composer. I am using the KubernetesPodOperator and would like to mount a google storage bucket to a directory in pod via gcsfuse. It seems like to do this I need to give k8s privileged security context as specified here. It seems like airflow recently added the security_context parameter to the KubernetesPodOperator. The security context I am specifying in the operator is :

security_context = {
  'securityContext': {
    'privileged': True,
    'capabilities':
      {'add': ['SYS_ADMIN']}
  }
}

When I try running airflow test dag_id task_id date in the airflow worker, the pod launches and when the code tries to mount the bucket via gcsfuse it throws the error "fusermount: fuse device not found, try 'modprobe fuse' first". This makes it seems as the security_context is not working (ex.).

Am I misunderstanding what the security_context parameter in the operator and/or is my securityContext dictionary definition invalid?

-- Nithin Saripalli
airflow
google-cloud-composer
google-kubernetes-engine
kubernetes

0 Answers