When my pod hits a failure state I'd like to have it deleted (and recreated by the deployment) rather than restarted. Is there a setting I could enable for that?
As mentioned by Alassane Ndiaye. Please take a look for "Voluntary and Involuntary Disruptions" and "Pod lifetime"
In general, Pods do not disappear until someone destroys them. This might be a human or a controller. The only exception to this rule is that Pods with a phase of Succeeded or Failed for more than some duration (determined by terminated-pod-gc-threshold in the master) will expire and be automatically destroyed.
All three types of controllers (Job, Replicaset, DaemonSet) contain a PodTemplate. It is recommended to create the appropriate controller and let it create Pods, rather than directly create Pods yourself. That is because Pods alone are not resilient to machine failures, but controllers are.
On the other hand if you are interested with more sophisticated control you should consider using Statefulset instead:
StatefulSets are valuable for applications that require one or more of the following:
- Stable, unique network identifiers.
- Stable, persistent storage
- Ordered, graceful deployment and scaling.
- Ordered, automated rolling updates.