I am running MySQL in Kubernetes with Kubernetes deployment. I have configured MySQL backup cronjob using https://github.com/databacker/mysql-backup

MySQL dump pod and MySQL pod are running in different nodes but in same AZ on AWS.

It was running fine for couple of months without any issues but last few days I see it is failing.

Sometimes there is nothing in the error logs and sometime I can see "Lost connection" in the log. I tried increasing the net_read_timeout value from 30 to 90. But this doesn't help.

I don't want to change different MySQL parameters as because with same setup MySQL backup is working fine in different region on another environment.

What will be correct path to go forward and troubleshoot this?