How to add dependency in flink minikube cluster?
I have a job jar which has dependency on flink-json-1.7.2.jar, flink-table_2.11-1.7.2.jar. This runs fine on my local machine if I add these two jars in lib folder of flink. How can I achieve the same on my minikube cluster. i.e. add the dependencies to flink. Ps. I have setup my minikube by following this documentation.
Also I can see those jars getting added in the fat jar created when I build the job jar.
Similar Questions
2 Answers
Due to an issue in Apache Calcite, which prevents the user classloaders from being garbage-collected, we do not recommend building a fat-jar that includes the flink-table dependency. Instead, we recommend configuring Flink to include the flink-table dependency in the system classloader. This can be done by copying the flink-table.jar file from the ./opt folder to the ./lib folder.
FROM docs
Since you have implemented Flink job on the local machine, I assume that it might be possible to create your own specific Docker image and include there requested *.jar files as described here. Although, you have already bootstrapped minikube Kubernetes cluster, customized Docker image can be used in the appropriate Flink job Kubernetes deployment files as well.