If I have multiple identical containers deployed simultaneously, and each contains a job to periodically create an artifact and save to a database, and what they save is deterministic, how should I go about preventing redundant operations?
Should I check the key in the database to see if it exists first, and if it doesn't, begin the saving operation? The artifact creation process is lengthy, so it's quite likely that one container may check the DB, see that it hasn't been saved to yet, and start the artifact creation process ... in the meantime, the other container may do the same.
I realize that having multiple clones of the same container is good for preventing downtime / keeping the application robust, but how should you deal with side effects?
This is a pretty open-ended question, so there isn't going to be one definitive answer without knowing the exact specifics of your situation.
Generally speaking in situations like this you should try to make the action that is being performed idempotent if possible, thus removing the issues if multiple requests are sent to perform the same action.
The question I would be asking myself is whether or not your architecture and technology stack is sutiable for this task. Not every activity needs to be performed in Kubernetes.
Would a Kubernetes CronJob be more sutiable for this?
What about a using messaging queue?