I'm working in a setup where I have a docker container running airflow deployed on Kubernetes. What I'm trying to do is package the dags definition file with the docker container that contains the airflow installation (for versioning purposes), and then have the ConfigMap which defines the dags_folder directory specify the directory (within airflow-docker) where that dags definition file is.
Dockerfile (airflow is the k8s namespace)
RUN mkdir /home/airflow/ \
&& mkdir /home/airflow/dags \
&& chown airflow:airflow /home/airflow \
&& chown airflow:airflow /home/airflow/dags
...
ADD dags.py /home/airflow/dags
USER airflowConfigMap
airflow.cfg: |
[core]
dags_folder = /home/airflow/dagsYou have to create config file
airflow.cfg: |
data:
core:
dags_folder = /home/airflow/dagsThen mount it in deployment.yml file.
apiVersion: apps/v1
kind: Deployment
metadata:
name: your_app_name
namespace: default or your namespace
spec:
replicas: 1
selector:
matchLabels:
app: your_app_name
template:
metadata:
labels:
app: your_app_name
spec:
container:
- name: Your image.
ports:
- name: my_app_port
containerPort: 7000
volumeMounts:
- mountPath: /your/directory/airflow.cfg
subPath: core
name: name of config_map # this should match with name of config map
volumes:
- name: name of the config_map
configMap:
name: your_config_map