I am a newbie to Kubernetes and have a design question for you.

There are 3 main things I am using:

1. Producer - the binary will be same but the sources from where the producer will be getting data are different. User can create any number of producers from different sources. They can be added or deleted at runtime.

2. Tasks - there are one or many tasks for one producer - they can be added or reduced at runtime. Every task runs some particular, unique query (defined by user).

3. A business logic entity - which will create new producers or new tasks OR delete producer(s) or task(s) depending on the user request.

Now, Currently I am achieving this by running business logic in a pod and it runtime creates producer pod(s) & task pod(s) by using “Kubernetes python client APIs”.

I would like to know is there any better way to do that?

I am definitely not looking for replicaSet or Horizontal Pod accelerator as the arguments to my producer/task pods will always change - depending from user data.

I am also okay with converting the business logic into any other Kubernetes entity which can runtime create/delete pods?

Any help would be appreciated.

Thank you.