We are currently building a multi-tenant cluster in EKS. For our persistent storage, we are using the Amazon EFS CSI driver https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html.
We are trying to use a single EFS with a different folder for each PV.
The issue we are facing is the security group needs to be applied at the node group level, so right now any pod with high enough privilege can just mount the EFS drive and delete anything on it.
Is there any way to resolve this security issue?
AFAIK, the only way is to make use UNIX style permissions in your EFS drive. Basically, you can create these permissions ahead of time for the subdirectories, based on a UNIX UID/GID, and then you can force your pods to start a certain UID/GID (and fsGroup) using a SecurityContext.
Additionally, you can also make use of MutatingAdminssionWebhooks so that your pods always get a securityContext automatically added based on say the namespace that they run on.