How to connect Zeppelin to Spark Master in Kubernetes

4/19/2018

I've installed Spark and Zeppeline on Kubernetes using helm chart incubator/Spark

In the Zeppeline interpreter, the Spark's master is set to local[*] Then, I changed to spark://ClusterIP_Spark_Master:7077

However, it seems that the interpreter cannot connect to Spark Master.

Is there any solution for this? Do I need to export SPARK_HOME?

Thanks

-- Evan
apache-spark
apache-zeppelin
kubernetes

1 Answer

4/19/2018

After I installed Spark 2.2.0 on docker image apache/zeppelin and set the SPARK_HOME to the Spark directory, I changed the url to spark://spark-master:7077.

Zeppelin can now connect to my Spark cluster.

The docker image I built with spark installed in /spark-2.2.0-bin-hadoop2.7

-- Evan
Source: StackOverflow