Managing volume rollbacks in K8s using persistent volumes

5/13/2021

I have a kubernetes deployment managed by a helm chart that I am planning an upgrade of. The app has 2 persistent volumes attached which are are EBS volumes in AWS. If the deployment goes wrong and needs rolling back I might also need to roll back the EBS volumes. How would one manage that in K8s? I can easily create the volume manually in AWS from my snapshot I've taken pre deployment but for the deployment to use it would I need to edit the pv yaml file to point to my new volume ID? Or would I need to create a new PV using the volume ID and a new PVC and then edit my deployment to use that claim name?

-- DeirdreRodgers
amazon-ebs
amazon-web-services
kubernetes
persistent-volumes

1 Answer

5/13/2021

First you need to define a storage class with reclaimPolicy: Delete

https://kubernetes.io/docs/concepts/storage/storage-classes/

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Delete
allowVolumeExpansion: true
mountOptions:
  - debug
volumeBindingMode: Immediate

Then, in your helm chart, you need to use that storage class. So, when you delete the helm chart, the persistent claim will be deleted and because the ReclaimPolicy=Delete for the storage class used, the corresponding persistent volume will also be deleted.

Be careful though. Once PV is deleted, you will not be able to recover that volume's data. There is no "recycle bin".

-- Rakesh Gupta
Source: StackOverflow