Trying to build the image from the provided download. Used "Source Code" package option download, not Hadoop versions. Following spark.apache.org/docs/2.3.0/running-on-kubernetes.html#docker-images first step to build a docker image. Have Docker CE with K8s running on Mac.
Using Terminal: Ran in spark folder:
./bin/docker-image-tool.sh -r cloud.docker.com/xxxx/spark buildAll runs until here:
Step 5/14 : COPY ${spark_jars} /opt/spark/jars COPY failed: stat /var/lib/docker/tmp/docker-builder840552377/assembly/target/scala-2.12/jars: no such file or directory
Please assist a newbie! Not sure where this folder structure is or whats up.
Please don't assume I did something correct....I could have missed an obvious install before running this. Help is appreciated after you laugh. :-)
Thanks
Make sure you've cloned the right spark project and then Build spark with this command
./build/mvn -Pkubernetes -DskipTests clean packageIf you just need to solve your current error, use this command (-pl is used to build a specified module where spark-assembly_2.11 is the artifactId as defined in assembly/pom.xml file.)
build/mvn -pl :spark-assembly_2.11 clean installThe scala jar will be available in the Assembly module folder.