Okay, so here's a little back story. I'm building a program that basically allows users to spin up their own little automation script. The script runs 24/7, its basically a continuous loop that depending on the user-set interval sometimes as low as 30 seconds, sends out an api call. That's about it.
Current program is written in node.js and currently what I do is whenever a user requests that their automation script start running I spawn a new child process using('child_process').fork, that basically encapsulates the required loop.
What I'm wondering is does google cloud provide the ability to handle this usecase? I understand i can just spin up a VM compute engine, but then scalability is limited to that VM. Any other solutions google cloud may offer that may facilitate this usecase?
A random idea, was to spin up docker containers instead of just forking into a new child process but is Container Engine be adequate for this, or does Container engine expect that you are scaling the same application (for ex. node web app api)
If the script only ever sends out API calls periodically you could drop the loop itself and replace it with delayed push queue tasks or cron jobs which send those API calls instead, meaning you can run this on App Engine, with automatic scalability.
Somehow related: High frequency data refresh with Google App Engine