Airflow with k8s executor - issues with PVC and reading hidden directory

2/22/2020

I am using a PVC to mount dags. The configs don't have examples and I am running into some issues.

## name of dags pvc
dags_volume_claim = dags_pvc

## I don't know what below is supposed to be
## if PVC is mounted at /opt/app and airflow is at /opt/app/airflow 
## dags should be in /opt/app/airflow/dags
## then does this sound right?
dags_volume_subpath = airflow/dags

Is the above setting correct?

Now the 2nd issue is that on this PVC I have all my dags, parsers and other scripts. There is a python script (dag) on this PVC under airflow directory that is supposed to access a file inside hidden directory on root of PVC.

So for us root of PVC (where PV is mounted) is

    /opt/app

The file is in

    /opt/app/.hidden/file1

When my webserver pod is coming up, it complains it cannot access

    /opt/app/.hidden/file1

I have it chmod to 777 for testing, but still not having any luck.

This is a python script if that helps.

Any ideas?

-- John Test
airflow
kubernetes
openshift

0 Answers