Redis seems to delete dump.rdb on startup. Using Kubernetes PVC's and KubeDB. Why is this happening?

6/28/2019

We are using KubeDB in our cluster to manage our DB's.

So Redis is deployed via a KubeDB Redis object and KubeDB attaches a PVC to the Redis pod.

Unfortunately KubeDB doesn't support any restoring or backing up of Redis dumps (yet).

For the backup our solution is to have a CronJob running which copies the dump.rdb from the Redis pod into the job pod and then uploads it to S3.

For the restoring of the dump I wanted to do the same, just the other way around. Have a temporary pod which downloads the S3 backup and then copies it over to the Redis pod into the dump.rdb location.

The redis.conf looks like this:

....
# The filename where to dump the DB
dbfilename dump.rdb

# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /data
....

The copying works. The dump.rdb is in the correct location with the correct permissions. I verified this by starting a second redis-server in the Redis pod using the same redis.conf. The dump.rdb is being loaded into the server without a problem.

However, since I don't want to manually start a second redis-server, I restarted the Redis pod (by kubectl delete pods) for the pod to pickup the copied dump.rdb.

Everytime I delete the pod, the dump.rdb is deleted and a new dump.rdb is being created with a much smaller size (93 bytes).

I don't believe it is a PVC issue since I have created a few files to test whether they are deleted as well. They are not. Only the dump.rdb.

Why does this happen? I am expecting Redis to just restore the DB from the dump.rdb and not create a new one.

EDIT: Yeah, size of dump.rdb is around 47 GB. Redis version is 4.0.11.

-- Moritz Schmitz v. Hülst
kubedb
kubernetes
kubernetes-pvc
redis

1 Answer

6/28/2019

Sooo, a few hours later, my teammate remembered that Redis executes a save to dump on shutdown.

Instead of deleting the pod using kubectl delete pod I now changed the code to run a SHUTDOWN NOSAVE using the redis-cli.

kubectl exec <redis-pod> -- /bin/bash -c 'redis-cli -a $(cat /usr/local/etc/redis/redis.conf | grep "requirepass " | sed -e "s/requirepass //g") SHUTDOWN NOSAVE'
-- Moritz Schmitz v. Hülst
Source: StackOverflow