Airflow PermissionError: [Errno 13] Permission denied: '/opt/airflow/logs/scheduler'

12/8/2021

I’m stuck with this issue, I trying to use persistent volume using NFS, but it doesn't work well the errors like this -->

File "/usr/local/lib/python3.8/pathlib.py", line 1292, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/local/lib/python3.8/pathlib.py", line 1288, in mkdir
    self._accessor.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/opt/airflow/logs/scheduler'

File "/usr/local/lib/python3.8/logging/config.py", line 570, in configure
    raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'processor

this is my configuration:

logs:
  path: '/opt/airflow/logs'
  persistence:
    enabled: true
    existingClaim: "pv-logs-claim"
    subPath: ""
    storageClass: "class-airflow-logs"
    accessMode: ReadWriteMany
    size: 10Gi

here my PV -->

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-logs
  labels:
    app: airflow
    release: airflow
spec:
  storageClassName: "class-airflow-logs"
  # storageClassName: "standard-rwo"
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  - ReadOnlyMany
  - ReadWriteMany
  nfs:
    path: /airflowfilestore
    server: xx.xx.xx.xx
  persistentVolumeReclaimPolicy: Retain

my PVC -->

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv-logs-claim
  labels:
    app: airflow
    release: airflow
spec:
  volumeName: pv-logs
  storageClassName: "class-airflow-logs"
  # storageClassName: "standard-rwo"
  accessModes:
  - ReadWriteOnce
  - ReadOnlyMany
  - ReadWriteMany
  resources:
    requests:
      storage: 50Gi

my class -->

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: class-airflow-logs
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-ssd
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=50000
  - gid=0
  - mfsymlinks
  - cache=strict
  - actimeo=30

The storage, persistent volume, persistent volume work well, the pv and pvc is already Bound, and i no idea why the pod doesn't have permission to write the logs

-- Ary Widiantara
airflow
kubernetes
kubernetes-helm
python

0 Answers