How to external access a ray remote multiple node cluster?

12/30/2019

I have a question concerning the access of a ray multi node cluster:

Considering a deployed cluster on AWS or kubernetes, how can a seperate python process from outside the cluster run tasks on the cluster?

e.g. A Web client tries to invoke python tasks,that should run within the Remote cluster.

EDIT: It comes down to the question:

Is there a Ray native API to connect to a remote cluster from a computer/server outside of the actual cluster?

-- Jwf
amazon-web-services
kubernetes
python
ray
remote-access

1 Answer

1/1/2020

https://ray.readthedocs.io/en/latest/deploy-on-kubernetes.html#running-ray-programs says you can get a shell on the ray pods and run the tasks or create a new Kubernetes job.

So the question boils down to how do I connect to the Kubernetes cluster from a web application.

You can connect to the Kubernetes cluster from a web application using one of the clients in https://github.com/kubernetes-client. Here is the Python one https://github.com/kubernetes-client/python

-- Tummala Dhanvi
Source: StackOverflow