Spark-2.4.0 docker build fails

1/30/2019

I am trying to build a spark container based on alpine linux (using provided Dockerfile in the distribution).

docker build -t spark:latest -f kubernetes/dockerfiles/spark/Dockerfile .

I see the following errors. How do I get past these problems?

Sending build context to Docker daemon  256.2MB
Step 1/15 : FROM openjdk:8-alpine
 ---> 2cfb1dc1f0c8
Step 2/15 : ARG spark_jars=jars
 ---> Using cache
 ---> 3929d9e6b4a3
Step 3/15 : ARG img_path=kubernetes/dockerfiles
 ---> Using cache
 ---> 74e046503051
Step 4/15 : ARG k8s_tests=kubernetes/tests
 ---> Using cache
 ---> b935956a4bd1
Step 5/15 : RUN set -ex &&     apk upgrade --no-cache &&     apk add --no-cache bash tini libc6-compat linux-pam &&     mkdir -p /opt/spark &&     mkdir -p /opt/spark/work-dir &&     touch /opt/spark/RELEASE &&     rm /bin/sh &&     ln -sv /bin/bash /bin/sh &&     echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su &&     chgrp root /etc/passwd && chmod ug+rw /etc/passwd
 ---> Running in 04372cfa8170
+ apk upgrade --no-cache
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
OK: 101 MiB in 53 packages
+ apk add --no-cache bash tini libc6-compat linux-pam
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
ERROR: unsatisfiable constraints:
  bash (missing):
    required by: world[bash]
  libc6-compat (missing):
    required by: world[libc6-compat]
  linux-pam (missing):
    required by: world[linux-pam]
  tini (missing):
    required by: world[tini]
The command '/bin/sh -c set -ex &&     apk upgrade --no-cache &&     apk add --no-cache bash tini libc6-compat linux-pam &&     mkdir -p /opt/spark &&     mkdir -p /opt/spark/work-dir &&     touch /opt/spark/RELEASE &&     rm /bin/sh &&     ln -sv /bin/bash /bin/sh &&     echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su &&     chgrp root /etc/passwd && chmod ug+rw /etc/passwd' returned a non-zero code: 4
-- Bhaskar
apache-spark-2.0
docker
kubernetes
linux

1 Answer

1/31/2019

It would be better for you if you run your preferred Linux (Alpine in your case) first as plain container and then exec to it.

Then try running your build commands one by one in container, then add it to your Dockerfile if you succeed.

Also as @wuerzelchen mentioned, not sure if Alpine is right choice for you as Linux distro.

-- coolinuxoid
Source: StackOverflow