Install Custom Connector To Kafka Connect on Kubernetes

5/21/2019

I'm running the kafka kubenetes helm deployment, however I am unsure about how to install a custom plugin.

When running custom plugin on my local version of kafka I mount the volume /myplugin to the Docker image, and then set the plugin path environment variable.

I am unsure about how to apply this workflow to the helm charts / kubernetes deployment, mainly how to go about mounting the plugin to the Kafka Connect pod such that it can be found in the default plugin.path=/usr/share/java.

-- Sam Palmer
apache-kafka
apache-kafka-connect
kubernetes

1 Answer

5/21/2019

Have a look at the last few slides of https://talks.rmoff.net/QZ5nsS/from-zero-to-hero-with-kafka-connect. You can mount your plugins but the best way is to either build a new image to extend the cp-kafka-connect-base, or to install the plugin at runtime - both using Confluent Hub.

-- Robin Moffatt
Source: StackOverflow