I installed the EFS CI driver and got their Static Provisioning example to work: I was able to start a pod that appended to a file on the EFS volume. I could delete the pod and start another one to inspect that file and confirm the data written by the first pod was still there. But what I actually need to do is mount the volume read-only, and I am having no luck there.

Note that after I successfully ran that example, I launched an EC2 instance and in it, I mounted the EFS filesystem, then added the data that my pods need to access in a read-only fashion. Then I unmounted the EFS filesystem and terminated the instance.

Using the configuration below, which is based on the Static Provisioning example referenced above, my pod does not start Running; it remains in ContainerCreating.

Storage class:

$ kubectl get sc efs-sc -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"efs-sc"},"provisioner":"efs.csi.aws.com"}
  creationTimestamp: "2020-01-12T05:36:13Z"
  name: efs-sc
  resourceVersion: "809880"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/efs-sc
  uid: 71ecce62-34fd-11ea-8a5f-124f4ee64e8d
provisioner: efs.csi.aws.com
reclaimPolicy: Delete
volumeBindingMode: Immediate

Persistent Volume (this is the only PV in the cluster that uses the EFS Storage Class):

$ kubectl get pv efs-pv-ro -o yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"efs-pv-ro"},"spec":{"accessModes":["ReadOnlyMany"],"capacity":{"storage":"5Gi"},"csi":{"driver":"efs.csi.aws.com","volumeHandle":"fs-26120da7"},"persistentVolumeReclaimPolicy":"Retain","storageClassName":"efs-sc","volumeMode":"Filesystem"}}
    pv.kubernetes.io/bound-by-controller: "yes"
  creationTimestamp: "2020-01-12T05:36:59Z"
  finalizers:
  - kubernetes.io/pv-protection
  name: efs-pv-ro
  resourceVersion: "810231"
  selfLink: /api/v1/persistentvolumes/efs-pv-ro
  uid: 8d54a80e-34fd-11ea-8a5f-124f4ee64e8d
spec:
  accessModes:
  - ReadOnlyMany
  capacity:
    storage: 5Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: efs-claim-ro
    namespace: default
    resourceVersion: "810229"
    uid: e0498cae-34fd-11ea-8a5f-124f4ee64e8d
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-26120da7
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  volumeMode: Filesystem
status:
  phase: Bound

Persistent Volume Claim (this is the only PVC in the cluster attempting to use the EFS storage class:

$ kubectl get pvc efs-claim-ro -o yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"efs-claim-ro","namespace":"default"},"spec":{"accessModes":["ReadOnlyMany"],"resources":{"requests":{"storage":"5Gi"}},"storageClassName":"efs-sc"}}
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
  creationTimestamp: "2020-01-12T05:39:18Z"
  finalizers:
  - kubernetes.io/pvc-protection
  name: efs-claim-ro
  namespace: default
  resourceVersion: "810234"
  selfLink: /api/v1/namespaces/default/persistentvolumeclaims/efs-claim-ro
  uid: e0498cae-34fd-11ea-8a5f-124f4ee64e8d
spec:
  accessModes:
  - ReadOnlyMany
  resources:
    requests:
      storage: 5Gi
  storageClassName: efs-sc
  volumeMode: Filesystem
  volumeName: efs-pv-ro
status:
  accessModes:
  - ReadOnlyMany
  capacity:
    storage: 5Gi
  phase: Bound

And here is the Pod. It remains in ContainerCreating and does not switch to Running:

$ kubectl get pod efs-app -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"efs-app","namespace":"default"},"spec":{"containers":[{"args":["infinity"],"command":["sleep"],"image":"centos","name":"app","volumeMounts":[{"mountPath":"/data","name":"persistent-storage","subPath":"mmad"}]}],"volumes":[{"name":"persistent-storage","persistentVolumeClaim":{"claimName":"efs-claim-ro"}}]}}
    kubernetes.io/psp: eks.privileged
  creationTimestamp: "2020-01-12T06:07:08Z"
  name: efs-app
  namespace: default
  resourceVersion: "813420"
  selfLink: /api/v1/namespaces/default/pods/efs-app
  uid: c3b8421b-3501-11ea-b164-0a9483e894ed
spec:
  containers:
  - args:
    - infinity
    command:
    - sleep
    image: centos
    imagePullPolicy: Always
    name: app
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /data
      name: persistent-storage
      subPath: mmad
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-z97dh
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: ip-192-168-254-51.ec2.internal
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim-ro
  - name: default-token-z97dh
    secret:
      defaultMode: 420
      secretName: default-token-z97dh
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2020-01-12T06:07:08Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2020-01-12T06:07:08Z"
    message: 'containers with unready status: [app]'
    reason: ContainersNotReady
    status: "False"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2020-01-12T06:07:08Z"
    message: 'containers with unready status: [app]'
    reason: ContainersNotReady
    status: "False"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2020-01-12T06:07:08Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - image: centos
    imageID: ""
    lastState: {}
    name: app
    ready: false
    restartCount: 0
    state:
      waiting:
        reason: ContainerCreating
  hostIP: 192.168.254.51
  phase: Pending
  qosClass: BestEffort
  startTime: "2020-01-12T06:07:08Z"

I am not sure if subPath will work with this configuration or not, but the same problem happens whether or not subPath is in the Pod configuration.

The problem does seem to be with the volume. If I comment out the volumes and volumeMounts section, the pod runs.

It seems that the PVC has bound with the correct PV, but the pod is not starting. I'm not seeing a clue in any of the output above, but maybe I'm missing something?

Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.8", GitCommit:"211047e9a1922595eaa3a1127ed365e9299a6c23", GitTreeState:"clean", BuildDate:"2019-10-15T12:11:03Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.9-eks-c0eccc", GitCommit:"c0eccca51d7500bb03b2f163dd8d534ffeb2f7a2", GitTreeState:"clean", BuildDate:"2019-12-22T23:14:11Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

aws-efs-csi-driver version: v.0.2.0.