Spark job on Kubernetes failing to launch

5/21/2018

I downloaded spark binaries, I created a docker image of dockerfile present under /kubernetes/dockerfiles and followed this documentation - https://spark.apache.org/docs/latest/running-on-kubernetes.html

I am trying to submit a spark job as follows -

    ~/spark-2.3.0-bin-hadoop2.7/bin/spark-submit     --master 
    k8s://http://<k8s-api-server>:<k8s-port>     --deploy-mode cluster     -- 
    name spark-pi     --class org.apache.spark.examples.SparkPi      -- 
    conf spark.executor.instances=5     --conf 
    spark.kubernetes.container.image=spark:1.0 --conf 
    spark.kubernetes.authenticate.driver.serviceAccountName=spark 
    local:///opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar

I am getting the following error -

++ id -u
+ myuid=0
++ id -g
+ mygid=0
++ getent passwd 0
+ uidentry=root:x:0:0:root:/root:/bin/ash
+ '[' -z root:x:0:0:root:/root:/bin/ash ']'
+ SPARK_K8S_CMD=driver
+ '[' -z driver ']'
+ shift 1
+ SPARK_CLASSPATH=':/opt/spark/jars/*'
+ env
+ grep SPARK_JAVA_OPT_
+ sed 's/[^=]*=\(.*\)/\1/g'
+ readarray -t SPARK_JAVA_OPTS
+ '[' -n /opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar:/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar ']'
+ SPARK_CLASSPATH=':/opt/spark/jars/*:/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar:/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar'
+ '[' -n '' ']'
+ case "$SPARK_K8S_CMD" in
+ CMD=(${JAVA_HOME}/bin/java "${SPARK_JAVA_OPTS[@]}" -cp "$SPARK_CLASSPATH" -Xms$SPARK_DRIVER_MEMORY -Xmx$SPARK_DRIVER_MEMORY -Dspark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS $SPARK_DRIVER_CLASS $SPARK_DRIVER_ARGS)
+ exec /sbin/tini -s -- /usr/lib/jvm/java-1.8-openjdk/bin/java -Dspark.driver.host=spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver-svc.default.svc -Dspark.kubernetes.container.image=spark:1.0 -Dspark.driver.port=7078 -Dspark.kubernetes.executor.podNamePrefix=spark-pi-560b11363c2f360b8d13f1d56eb3966c -Dspark.driver.blockManager.port=7079 -Dspark.kubernetes.driver.pod.name=spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver -Dspark.kubernetes.authenticate.driver.serviceAccountName=spark -Dspark.executor.instances=5 -Dspark.app.name=spark-pi -Dspark.master=k8s://http://10.118.202.130:8080 -Dspark.submit.deployMode=cluster -Dspark.jars=/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar,/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar -Dspark.app.id=spark-810f3f33b27844da9f584bb4ef661f45 -cp ':/opt/spark/jars/*:/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar:/opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar' -Xms1g -Xmx1g -Dspark.driver.bindAddress=10.118.202.118 org.apache.spark.examples.SparkPi
2018-05-21 20:59:30 INFO  SparkContext:54 - Running Spark version 2.3.0
2018-05-21 20:59:31 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-05-21 20:59:31 INFO  SparkContext:54 - Submitted application: Spark Pi
2018-05-21 20:59:31 INFO  SecurityManager:54 - Changing view acls to: root
2018-05-21 20:59:31 INFO  SecurityManager:54 - Changing modify acls to: root
2018-05-21 20:59:31 INFO  SecurityManager:54 - Changing view acls groups to:
2018-05-21 20:59:31 INFO  SecurityManager:54 - Changing modify acls groups to:
2018-05-21 20:59:31 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2018-05-21 20:59:31 INFO  Utils:54 - Successfully started service 'sparkDriver' on port 7078.
2018-05-21 20:59:31 INFO  SparkEnv:54 - Registering MapOutputTracker
2018-05-21 20:59:31 INFO  SparkEnv:54 - Registering BlockManagerMaster
2018-05-21 20:59:31 INFO  BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
2018-05-21 20:59:31 INFO  BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up
2018-05-21 20:59:31 INFO  DiskBlockManager:54 - Created local directory at /tmp/blockmgr-10dc281e-86b2-4f82-ae46-531ee0db01ee
2018-05-21 20:59:31 INFO  MemoryStore:54 - MemoryStore started with capacity 408.9 MB
2018-05-21 20:59:31 INFO  SparkEnv:54 - Registering OutputCommitCoordinator
2018-05-21 20:59:31 INFO  log:192 - Logging initialized @1651ms
2018-05-21 20:59:31 INFO  Server:346 - jetty-9.3.z-SNAPSHOT
2018-05-21 20:59:32 INFO  Server:414 - Started @1729ms
2018-05-21 20:59:32 INFO  AbstractConnector:278 - Started ServerConnector@66629f63{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2018-05-21 20:59:32 INFO  Utils:54 - Successfully started service 'SparkUI' on port 4040.
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@493dfb8e{/jobs,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@15a902e7{/jobs/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@7876d598{/jobs/job,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5af28b27{/jobs/job/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@71104a4{/stages,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4985cbcb{/stages/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@72f46e16{/stages/stage,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@549621f3{/stages/stage/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@54361a9{/stages/pool,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@32232e55{/stages/pool/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5217f3d0{/storage,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@37ebc9d8{/storage/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@293bb8a5{/storage/rdd,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@2416a51{/storage/rdd/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6fa590ba{/environment,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6e9319f{/environment/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@72e34f77{/executors,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@7bf9b098{/executors/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@389adf1d{/executors/threadDump,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@77307458{/executors/threadDump/json,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@1fc0053e{/static,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@ec0c838{/,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6e46d9f4{/api,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@7997b197{/jobs/job/kill,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@11dee337{/stages/stage/kill,null,AVAILABLE,@Spark}
2018-05-21 20:59:32 INFO  SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver-svc.default.svc:4040
2018-05-21 20:59:32 INFO  SparkContext:54 - Added JAR /opt/spark/examples/jars/spark-examples_2.11-2.3.0.jar at spark://spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver-svc.default.svc:7078/jars/spark-examples_2.11-2.3.0.jar with timestamp 1526936372118
2018-05-21 20:59:32 WARN  KubernetesClusterManager:66 - The executor's init-container config map is not specified. Executors will therefore not attempt to fetch remote or submitted dependencies.
2018-05-21 20:59:32 WARN  KubernetesClusterManager:66 - The executor's init-container config map key is not specified. Executors will therefore not attempt to fetch remote or submitted dependencies.
2018-05-21 20:59:42 ERROR SparkContext:91 - Error initializing SparkContext.
org.apache.spark.SparkException: External scheduler cannot be instantiated
    at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$createTaskScheduler(SparkContext.scala:2747)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:492)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2486)
    at org.apache.spark.sql.SparkSession$Builder$anonfun$7.apply(SparkSession.scala:930)
    at org.apache.spark.sql.SparkSession$Builder$anonfun$7.apply(SparkSession.scala:921)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
    at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31)
    at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get]  for kind: [Pod]  with name: [spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver]  in namespace: [default]  failed.
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62)
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:71)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:228)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:184)
    at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend.<init>(KubernetesClusterSchedulerBackend.scala:70)
    at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:120)
    at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$createTaskScheduler(SparkContext.scala:2741)
    ... 8 more
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
    at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:281)
    at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:251)
    at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:151)
    at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:195)
    at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
    at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at io.fabric8.kubernetes.client.utils.HttpClientUtils$2.intercept(HttpClientUtils.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
    at okhttp3.RealCall.execute(RealCall.java:69)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:377)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:312)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:295)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:783)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:217)
    ... 12 more
    2018-05-21 20:59:42 INFO  AbstractConnector:318 - Stopped 
    Spark@66629f63{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
    2018-05-21 20:59:42 INFO  SparkUI:54 - Stopped Spark web UI at http://spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver-svc.default.svc:4040
    2018-05-21 20:59:42 INFO  MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!
    2018-05-21 20:59:42 INFO  MemoryStore:54 - MemoryStore cleared
    2018-05-21 20:59:42 INFO  BlockManager:54 - BlockManager stopped
    2018-05-21 20:59:42 INFO  BlockManagerMaster:54 - BlockManagerMaster stopped
    2018-05-21 20:59:42 WARN  MetricsSystem:66 - Stopping a MetricsSystem that is not running
    2018-05-21 20:59:42 INFO  OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
    2018-05-21 20:59:42 INFO  SparkContext:54 - Successfully stopped SparkContext
    Exception in thread "main" org.apache.spark.SparkException: External scheduler cannot be instantiated
        at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$createTaskScheduler(SparkContext.scala:2747)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:492)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2486)
        at org.apache.spark.sql.SparkSession$Builder$anonfun$7.apply(SparkSession.scala:930)
        at org.apache.spark.sql.SparkSession$Builder$anonfun$7.apply(SparkSession.scala:921)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
        at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31)
        at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
    Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get]  for kind: [Pod]  with name: [spark-pi-560b11363c2f360b8d13f1d56eb3966c-driver]  in namespace: [default]  failed.
        at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62)
        at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:71)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:228)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:184)
        at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend.<init>(KubernetesClusterSchedulerBackend.scala:70)
        at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:120)
        at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$createTaskScheduler(SparkContext.scala:2741)
        ... 8 more
    Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
        at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:281)
        at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:251)
        at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:151)
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:195)
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at io.fabric8.kubernetes.client.utils.HttpClientUtils$2.intercept(HttpClientUtils.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
        at okhttp3.RealCall.execute(RealCall.java:69)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:377)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:312)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:295)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:783)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:217)
        ... 12 more
    2018-05-21 20:59:42 INFO  ShutdownHookManager:54 - Shutdown hook called
    2018-05-21 20:59:42 INFO  ShutdownHookManager:54 - Deleting directory 
    /tmp/spark-286e2bfe-9a7a-490a-aade-88fd81ff49c4

Please help.

Thanks.

-- codechef
apache-spark
gpu
kubernetes

1 Answer

5/22/2018

Im just guessing here, is your kube cluster running with rbac enabled? If yes, did you apply the rbac ?

https://spark.apache.org/docs/latest/running-on-kubernetes.html#rbac

-- Bal Chua
Source: StackOverflow