I have code that takes messages from a queue and does work. The work is mostly I/O bound. I need to be able to scale the number of workers.
I can write an application that manages threads or uses TPL to scale and manage the workers. This works but overtime has proven to be a bad approach on many fronts.
Comparing these 3 options is it possible to rank them in terms of how much resource they consume for the same amount of workers?
1) write a simple application and run many instances.
2) deploy the simple application in Service Fabric and scale it.
3) do something similar to 2) with Kubernetes.
Do others use this approach?
I know there are other factors but I'm focused on the resource usage comparison.
There are many approaches, and most of these is already answered on similar stackoverflow questions.
Each of these will have good and bad points to consider, and only doing a deep analysis on your solution using then to chose the right one. What I can recommend is doing a proof of concept for each approach and check which one suits well your application.
I recommend you take a look on these to start with:
Simulate 10,000 Azure IoT Hub Device connections from Azure Service Fabric cluster
What is the scenario for deploying multiple applications in service fabric?
And on this github issue with a few discussion points.