I need to run some dockers in map-reduce mode:
For example:
The list [[100, 120, 130], [155, 156], [180, 191, 197, 199]]
will be splitted to 3 different dockers, each docker will get different list value
The results of the 3 dockers will be used as input to new docker (reduce mode).
https://medium.com/better-programming/running-a-mapreduce-word-count-application-in-docker-using-python-sdk-ed2f1e27d424
which suggest similar problem. The author wrote code (python
using docker
package) for map-reduce and implemented it.
Is there any framework which do this dockers map-reduce ? or
I need to implement it ?