Implementing a no-downtime upgrade in Kubernetes

7/24/2017

I'm trying to deploy a Piwik site onto Kubernetes, and I am struggling to figure out how to do a hot-database-upgrade without loosing stats. So the process is as follows(upgrading from Piwik 3.0 to 3.1):

  1. Spin up redis
  2. Spin up Piwik 3.0 in maintenance mode which will stop it from connecting to the database
  3. Spin down old Piwik 3.0(in regular mode)
  4. Spin up Piwik 3.1(in maintenance mode)
  5. Run the database migrations.
  6. Spin down old Piwik 3.0(in maintenance mode)
  7. Spin up Piwik 3.1(in regular mode)
  8. Spin down Piwik 3.1(in maintenance mode)

Can I script this inside Kubernetes, or is my best approach going to be running a script inside my master. My aim is to be able to do repeatedly and reliably(so failures prior to step 5 are rolled back, and failures afterwards rely on human intervention).

I have tried to do some research around the best approach but I cannot find much information regarding this sort of upgrade process. Most guides just recommend taking the service off-line for a few seconds/minutes to do the migration which in this application isn't acceptable.

-- Opal
database-update
deployment
kubernetes

1 Answer

7/25/2017

If your instance of Piwik was created as a Deployment, you're in luck. The details are here. What you need to do is a RollingUpdate.

The process will look like this. The syntax is "Action: Result".

  1. Enable maintenance mode in Piwik Deployment: k8s progressively starts new Pods of Piwik 3.0 in maintenance mode, and spin down old Piwik 3.0. This occurs Pod by Pod.
  2. Update image tag in Piwik Deployment to 3.1: k8s progressively start new Pods of Piwik 3.1, and spin down Piwik 3.0 maintenance mode. This occurs Pod by Pod.
  3. Run the database migrations: Do your schtuff.
  4. Disable maintenance mode in Piwik Deployment: k8s progressively start new Pods of Piwik 3.1 regular mode, and spin down Piwik 3.1 maintenance mode. This occurs Pod by Pod.
-- Eugene Chow
Source: StackOverflow