I have deployed "spark-hadoop" docker image on the docker container in minikube.
I have created kubectl services on Kubernetes for the spark components like spark-master, spark-worker and spark-ui. I have exposed spark-master as a KubeService via NodePort and hence i can access the spark-webui page as:
http://{minikubeIP}:{Nodeport}
from my chrome browser inside minikube
The sparkui webpage at http://{minikubeIP}:{Nodeport} on the broswer displays the typical spark job dashboard and also mentioned that spark-master is alive and running at "spark://spark-master:7077".
Now the issue is how should i access "spark-master" URL from my another docker containerized application. In this other application i have mentioned {spark.master=spark://spark-master:7077"} in application.properties. When starting up the application gives error like "Master not responsive" and it shuts down.
Any pointers on this will help.
minispark@minispark-VirtualBox: kubectl describe svc spark-master
Name: spark-master
Namespace: eniq-dev
Labels: <none>
Annotations: <none>
Selector: component=spark-master
Type: NodePort
IP: 10.97.44.8
Port: webui 8080/TCP
TargetPort: 8080/TCP
NodePort: webui 32396/TCP
Endpoints: 172.18.0.8:8080
Port: spark 7077/TCP
TargetPort: 7077/TCP
NodePort: spark 31599/TCP
Endpoints: 172.18.0.8:7077
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>As for the pure service approach, no it is not possible. Service relates to only one selector so you can't assign another application in different pod to the same service.
If you are running two containers in the same pod use different ports inside the pod or use separate pods. Create deployment if there is no shared resources like filesystem - it would be easy to split the containers to separate pods.
Note that it will not suffice to change the pod definition if you want to have both containers running in the same pod with different ports. The application in the container must bind to a different port as well.
However best solution is to create another service which will point to different application and access it via different url.
Take a look: single-service-for-multiple-pods, how-to-configure-multiple-services-containers-in-kubernetes, multi-container-pods-communication, understanding-multi-container-pods.