kubernetes orchestrate different images on a worker fleet

12/1/2021

I have a use case to deploy different connectors on a sink fleet to port data from kafka to different destinations(es, another Kafka, databases. etc). Kafka has those connectors in place and I am wondering how to orchestrate those connector images in the same fleet based on demand. say if i want to direct data for topicA to es and for topicB to hdfs, an orchestrator will read the above two tasks and spin up workers to perform the data transfer to different destinations. but I don't know if kubernetes has builtin feature or plug-ins to act as the orchestrator above?

-- dashenswen
apache-kafka
kubernetes

0 Answers