My DAG is getting stuck in running state with a log message "had an event of type Pending" in stack driver. My DAG works fine without volume mount but looks like there is an issue with volume mount. Can you please someone help here?
I am trying to mount "/data/storage" path to "/storage" location in container.
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.contrib.kubernetes.volume import Volume
from airflow.contrib.kubernetes.volume_mount import VolumeMount
from datetime import timedelta
default_args = {
'owner': '',
'start_date': '2020-03-31 03:40:34',
'depend_on_past': False,
'email_on_failure': False,
'email_on_retry': False,
'retries': 0,
'retry_delay': timedelta(minutes=5)
}
volume_mount = VolumeMount('test-volume',
mount_path='/data/storage',
sub_path='storage',
read_only=True)
volume_config= {
'persistentVolumeClaim':
{
'claimName': 'test-volume'
}
}
volume = Volume(name='test-volume', configs=volume_config)
dag = DAG('dag-name', default_args=default_args, schedule_interval=None, catchup=False, concurrency=1, max_active_runs=1)
task1 = KubernetesPodOperator(
# The ID specified for the task.
task_id='task1',
# Name of task you want to run, used to generate Pod ID.
name='task1',
namespace='default',
env_vars={'var1': 'value1'},
image='someimage',
volumes=[volume],
volume_mounts=[volume_mount],
arguments=['arg.json'],
startup_timeout_seconds=3600,
image_pull_policy='Always',
dag=dag)
Since you already define the mount_path='/data/storage'
I don't think it's necessary to add sub_path='storage'
. We can see it both in the docstring provided in here as well as the example provided in here.
Have you looked at this post? first you need to create a persistent volume and then update your DAG with a volume and a volume mount. Here is the full example