I'm trying to figure out a portable way to develop a custom but scalable task queue in my cluster for google container engine . This is the scenario I have a front end that captures users details in my node js instance ,these details are sent to the api system which in turn contacts the db ,saves the user details and is expected to send a welcome mail . My issue is this i don't want to use the same api endpoint method to process the sending mail requests ,I need another process to handle that how do I handle that with my kubernetes infrastructure?.Do I need to implement a pub sub type of system to publish to another container ?.If I do this it means all subscribes will be notified of my update but what if I have 2 instances of my sub system running it means they will all observe the changes and send the mail twice. Any thoughts or ideas on this would be appreciated.
I see two reasonable ways to approach this.
1: have a service that takes in mailing events by means of an API and returns immediately after receiving to process mailing asynchornously. Using kube service you will hit only one such service and one mail will be sent in a non blocking way for the calling service, but it has downsides - ie. what happens if something fails, the mail might not be generated at all.
2: I would go for some MQ probably (Kafka, Rabbit etc.), have a message queue consumed by any number of mailing service instances, make sure that only one can pick up the message, and require an ack for the message or return it to processing if no ack in N min