Not able to run airflow in Cloud run getting error disk I/o error

4/5/2020

I am trying to run airflow in google cloud run. Getting error Disk I/O error, I guess the disk write permission is missing.

can someone please help me with this how to give write permission inside cloud run. I also have to write file and later delete it. Error in cloud run logs

-- Deepak Poojari
airflow
google-cloud-platform
google-cloud-run
google-kubernetes-engine

1 Answer

4/5/2020

Only the directory /tmp is writable in Cloud Run. So, change the default write location to write into this directory.

However, you have to be aware of 2 things:

  • Cloud Run is stateless, that means when a new instance is created, the container start from scratch, with an empty /tmp directory
  • /tmp directory is an in-memory file system. The maximum allowed memory on Cloud Run is 2Gb, your app memory footprint included. In addition of your file and Airflow, not sure that you will have a lot of space.

A final remark. Cloud Run is active only when it process request, and a request has a maximum timeout of 15 minutes. When no request, the allowed cpu is close to 0%. I'm not sure of what you want to achieve with Airflow on Cloud Run, but my feeling tells me that your design is strange. And I prefer to warn you before you spend too much effort on this.

-- guillaume blaquiere
Source: StackOverflow