How to call python utility files with fuction in k8podoperator tried but not working p_utils.initiate_batch

5/25/2021
<!-- begin snippet: js hide: false console: true babel: false --><!-- language: lang-js -->
p_utils = imp.load_source(u'p_utils', join('/efs/com/cm/utilities/cm_integration_python_utils.py'))

#######Python operator DAg code to run utility file with spaecific function within that python utility############ lash_site_init_task = PythonOperator(task_id='xyz',python_callable=p_utils.initiate_batch, op_kwargs={'source':'lash','feed':'site'}, trigger_rule=TriggerRule.ALL_SUCCESS,dag=lash_dag)

-----------------------------------------------
######Convert pythonoperator using k8podopearator 

lash_site_init_task = KubernetesPodOperator(namespace=namespace, image=image ,cmds=["/bin/bash","-c", p_utils.compare_lst_modified_time],op_kwargs={'source':'lash','feed':'prvdr'}, volume_mounts=[volume_mount], volumes=[volume],name="table-lash_prvdr_data_load_check", task_id='lash_prvdr_data_load_check',get_logs=True, provide_context=True, config_file=kube_config_path, in_cluster=False, cluster_context='aws',
                                            trigger_rule=TriggerRule.ALL_SUCCESS,
                                                 on_failure_callback=p_utils.update_audit_failed_status('lash', 'prvdr', 'data_load_check'),dag=lash_dag)
<!-- end snippet -->
-- jayesh_mahajan
airflow
kubernetes
kubernetes-pod
python-2.7
utility

0 Answers