How to run multiple simulations inside containers programmatically?

8/19/2020

I am developing a website that runs a simulation given a user-submitted script. I tried to follow some Online Judge architectures, but in my case, I need to send user input and receive the output in realtime, like a simulation.

I tried Kubernetes Jobs, but isn't seems so easy to communicate with the container, especially if I need a Kubernetes client on the language that I am working.

So, my question is: Given this scenario, what is the best approach to orchestrate multiple containers with interactive I/O programmatically?

Diagram

*Obs.: I am not worrying about security yet.

-- Ícaro Lima
amazon-ecs
amazon-web-services
docker
docker-swarm
kubernetes

1 Answer

8/19/2020

Please take a look at the design of the spark operator:

https://github.com/GoogleCloudPlatform/spark-on-k8s-operator

That has a somewhat similar design to what you’re targeting. Similarly, Argo Workflow is another example:

https://github.com/argoproj/argo

-- Faheem
Source: StackOverflow