I want to run an app on any node. It should always have at least one instance per node, but more instances are allowed, primarily during an update to prevent downtime of that pod (and node).
Kubernetes deployment updates usually work by launching a new pod, and as soon as it is available the old one is terminated. That's perfect, but in my case I need a DaemonSet to launch a specific app on all nodes at all times. However, when updating this DaemonSet, Kubernetes kills a pod one by one (i.e. node by node) and then launches a new pod, which means that on any given time during an update the pod may not be running on a node.
It seems that DaemonSets are, compared to Deployments, the correct way to do that, but I couldn't find any way to prevent downtime when updating the DaemonSet. Is there any way to do this? I also thought of using Deployments and update a replica amount manuall and antiPodAffinity so only one pod gets deployed per node, but this is kind of hacky.
Assuming your application can tolerate having more than a single pod per node (which seems to be the case), you could create a second DaemonSet, and delete the original one once you're sure the new DaemonSet is functional. This process is a bit more involved, but it lets you control when the old pods are deleted.
Note that you'd end up with a DaemonSet under a different name. If that bothers you, you could always repeat the process once more to get back to the original name.
There was a very long set of discussions about adding this feature. You can see them here and here
Long story short, this isn't really possible. You can try and combine maxUnavailable: 0 and type: rollingUpdate in your updateStrategy but I don't think that's formally supported.
Example:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: my-daemonset
labels:
service: my-daemonset
spec:
selector:
matchLabels:
service: my-daemonset
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
template:
metadata:
labels:
service: my-daemonset
spec:
containers:
- name: daemonset-update
image: my-image:latestIt is possible to perform rolling updates with no downtime using a DeamonSet as of Today! What you need is to have at least 2 nodes running on your cluster and set maxUnavailable to 1 in your DaemonSet configuration.
Assuming the previous configuration, when an update is pushed, a first node will start updating. The second will waiting until the first completes. Upon success, the second does the same.
The major drawback is that you need to keep 2 nodes runnings continuously or to take actions to spawn/kill a node before/after an update.