I am trying to deploy a technology over Kubernetes that requires sharing private data between pods. In my environment, there is a Root CA that creates a .pem file that is needed by an Intermediate CA to connect, so it has to be created and shared. In addition, Root CA can change that .pem file periodically, so I just can not create the RCA, kubectl copy and then start the ICA with that data filled.
My first approach was creating a secret and keeping it updated from the RCA pod, but, as long as I have seen, a pod can not create a secret without having some admin permissions that I do not want at all in a production environment. I have also thought about using some NFS volume between them to share it, but publishing private data over NFS is not secure at all and keeping alive a NFS Volume/Server just to share one file sometimes looks like an overkill to me. SSH/SCP is also a solution I have thought about, but looks like it also has its own problems and I also have to had some ports open that I would not want to.
I have been stuck at this problem a lot of time and do not know how to solve it. I hope it gets clear enough to receive some help. Thank you.
My advice is to create a cronJob object that updates the secret using an image with kubectl. Also to ensure your pods uses the last CA file you can use an operator that watches for changes in the secret an ensures to reload the pods related to the CA file every time it changes.
An operator I've used in the past and works is Reloader, with that you can put an annotation on the pod linking the secret you need to watch.