I'd like to be able to create a temporary container within an existing pod to handle processing arbitrary code from a user (security concerns), however this container must also in the same pod due to data locality / performance concerns.
My question is, what is the proper way to achieve this? Ephemeral Containers are described as "to inspect services rather than to build applications" and "Ephemeral containers may not have ports".
So I feel that this is not the proper way to go about this. My temporary container must be able to share mounted data with the original container in the same Pod, and must be able to communicate via a port that is opened to the original container of the same Pod.
You can achieve this by either creating a sidecar which will intercept the traffic to your original pod, or just creating a second pod in your deployment and a way of automatically trigger the process you wanna do. One caveat about this is that the two pods share the same network so you can not expose the same port for both containers.
The downside of both approaches is that you no longer have a temporary container,now you would have both up and running.
If what you wanna do is a one time task when your container is up I highly recommend exposing an API in your original pod and make a call from a Job.