Flink Docker Image

12/9/2019

I am new in Docker, I am trying to create Flink docker image. Docker image is showing created successfully. But I have a doubt, I can not see Flink binaries in /opt path as mentioned in Dockerfile.

How do I know my Flink Docker image is created successfully.

Screenshot from console, few set of commands are highlighted in red color, means is it OK ? enter image description here

Status showing Flink Docker image is created: enter image description here

Please help me. Thank you....

Full Log:

    sudo /home/develk/cntx_eng/build.sh \
> --job-artifacts /home/develk/cntx_eng/FlinkContextEnginePoc-0.0.1-SNAPSHOT.jar \
> --from-archive /home/develk/cntx_eng/flink-1.4.0-bin-hadoop24-scala_2.11.tgz \
> --image-name contxeng-flink-poc:1.4.0
--job-artifacts
/home/develk/cntx_eng/FlinkContextEnginePoc-0.0.1-SNAPSHOT.jar
--from-archive
JOB_ARTIFACTS_PATH : /home/develk/cntx_eng/FlinkContextEnginePoc-0.0.1-SNAPSHOT.jar
FROM_ARCHIVE : /home/develk/cntx_eng/flink-1.4.0-bin-hadoop24-scala_2.11.tgz
HADOOP_VERSION :
FLINK_VERSION :
-------------------------Arg Values---------------------------
FLINK_DIST : _TMP_/flink.tgz
JOB_ARTIFACTS_TARGET : _TMP_/artifacts
SHADED_HADOOP :
IMAGE_NAME : contxeng-flink-poc:1.4.0
--------------------------------------------------------------
Sending build context to Docker daemon  606.6MB
Step 1/23 : FROM openjdk:8-jre-alpine
 ---> f7a292bbb70c
Step 2/23 : RUN apk add --no-cache bash snappy libc6-compat
 ---> Using cache
 ---> 9e84497f3616
Step 3/23 : ENV FLINK_INSTALL_PATH=/opt
 ---> Using cache
 ---> 87bc358ccf00
Step 4/23 : ENV FLINK_HOME $FLINK_INSTALL_PATH/flink
 ---> Using cache
 ---> 712ba8d54555
Step 5/23 : ENV FLINK_LIB_DIR $FLINK_HOME/lib
 ---> Using cache
 ---> 80e7b085252e
Step 6/23 : ENV FLINK_PLUGINS_DIR $FLINK_HOME/plugins
 ---> Using cache
 ---> 7d39101e47d3
Step 7/23 : ENV FLINK_OPT_DIR $FLINK_HOME/opt
 ---> Using cache
 ---> 9bff7fc7145d
Step 8/23 : ENV FLINK_JOB_ARTIFACTS_DIR $FLINK_INSTALL_PATH/artifacts
 ---> Using cache
 ---> b0c01f3aab84
Step 9/23 : ENV FLINK_USR_LIB_DIR $FLINK_HOME/usrlib
 ---> Using cache
 ---> f4236bc26cab
Step 10/23 : ENV PATH $PATH:$FLINK_HOME/bin
 ---> Using cache
 ---> 2cb7cd442b6f
Step 11/23 : ARG flink_dist=NOT_SET
 ---> Using cache
 ---> 1a6fc691baa2
Step 12/23 : ARG job_artifacts=NOT_SET
 ---> Using cache
 ---> e11400e03120
Step 13/23 : ARG python_version=NOT_SET
 ---> Using cache
 ---> 313089fd991e
Step 14/23 : ARG hadoop_jar=NOT_SET*
 ---> Using cache
 ---> ccbef4dfa806
Step 15/23 : RUN   if [ "$python_version" = "2" ]; then     apk add --no-cache python;   elif [ "$python_version" = "3" ]; then     apk add --no-cache python3 && ln -s /usr/bin/python3 /usr/bin/python;   fi
 ---> Using cache
 ---> 7e6dca36dad4
Step 16/23 : ADD $flink_dist $hadoop_jar $FLINK_INSTALL_PATH/
 ---> 5afb7a8e5414
Step 17/23 : ADD $job_artifacts/* $FLINK_JOB_ARTIFACTS_DIR/
 ---> c2789d3d80b3
Step 18/23 : RUN set -x &&   ln -s $FLINK_INSTALL_PATH/flink-[0-9]* $FLINK_HOME &&   ln -s $FLINK_JOB_ARTIFACTS_DIR $FLINK_USR_LIB_DIR &&   if [ -n "$python_version" ]; then ln -s $FLINK_OPT_DIR/flink-python*.jar $FLINK_LIB_DIR; fi &&   if [ -f ${FLINK_INSTALL_PATH}/flink-shaded-hadoop* ]; then ln -s ${FLINK_INSTALL_PATH}/flink-shaded-hadoop* $FLINK_LIB_DIR; fi &&   addgroup -S flink && adduser -D -S -H -G flink -h $FLINK_HOME flink &&   chown -R flink:flink ${FLINK_INSTALL_PATH}/flink-* &&   chown -R flink:flink ${FLINK_JOB_ARTIFACTS_DIR}/ &&   chown -h flink:flink $FLINK_HOME
 ---> Running in c4cb70216f08
+ ln -s /opt/flink-1.4.0 /opt/flink-1.4.0-bin-hadoop24-scala_2.11.tgz /opt/flink
+ ln -s /opt/artifacts /opt/flink/usrlib
+ '[' -n  ]
+ '[' -f /opt/flink-shaded-hadoop-2-uber-2.4.1-8.0.jar ]
+ ln -s /opt/flink-shaded-hadoop-2-uber-2.4.1-8.0.jar /opt/flink/lib
+ addgroup -S flink
+ adduser -D -S -H -G flink -h /opt/flink flink
+ chown -R flink:flink /opt/flink-1.4.0 /opt/flink-1.4.0-bin-hadoop24-scala_2.11.tgz /opt/flink-shaded-hadoop-2-uber-2.4.1-8.0.jar
+ chown -R flink:flink /opt/artifacts/
+ chown -h flink:flink /opt/flink
Removing intermediate container c4cb70216f08
 ---> 459b1156294b
Step 19/23 : COPY docker-entrypoint.sh /
 ---> d4ae4be34415
Step 20/23 : USER flink
 ---> Running in 95a2c9234cd5
Removing intermediate container 95a2c9234cd5
 ---> ebdc913c7dd9
Step 21/23 : EXPOSE 8081 6123
 ---> Running in f6fad553a1d7
Removing intermediate container f6fad553a1d7
 ---> 51e6c57d2bde
Step 22/23 : ENTRYPOINT ["/docker-entrypoint.sh"]
 ---> Running in 09e7c0759fb6
Removing intermediate container 09e7c0759fb6
 ---> 99cdeb095b8f
Step 23/23 : CMD ["--help"]
 ---> Running in f985bd546dcf
Removing intermediate container f985bd546dcf
 ---> 039086df61e6
Successfully built 039086df61e6
Successfully tagged contxeng-flink-poc:1.4.0
-- Atanu chatterjee
apache-flink
docker
dockerfile
kubernetes

1 Answer

12/9/2019

Seems like everything went well.

To see which containers are running:

docker ps

contxeng-flink-poc:1.4.0, should be in there.

If you actually want to interact with Flink you should expose some ports. The Flink dashboard for example runs on port 8081 of your container.

You can also get an interactive bash shell into your container by running the following command (if your container has bash installed):

docker exec -it contxeng-flink-poc:1.4.0 bash

That's where you will find your Flink binaries.

If the container is not running, check if it was built:

docker images

If that's the case, run it:

docker run -d contxeng-flink-poc:1.4.0
-- Quinten Scheppermans
Source: StackOverflow