Rails server on Kubernetes: Resque Job using stale Database

7/10/2019

I've got a Rails server on a Kubernetes cluster. It uses a Mysql backend. We've got a class that does something like:

  1. Create Database Object
  2. Run Resque Job, params = Database Object ID

and the Job

  1. Grab Database Object from ID
  2. Do something with object.

However, the job fails to find the database record!

We tried several things to investigate the problem

  1. We added a sleep(1) command inside the job before it looks for the record. This fixes the problem
  2. We investigated whether this was a cache issue, however clearing the cache did not fix the problem.
  3. We added logging: We logged DatabaseModel.last.id in the class and in the job. In the class it would return 1000 and in the job it would return 999. i.e. the record had not yet reflected in the database for the job for some reason.

As far as I understand there is only one copy of the database on the server so I do not understand how this could be happening. This may be a bit of a reach for someone to be able to help without more context, but it's worth a try :)

thank you

-- Makoto
kubernetes
mysql
resque
ruby-on-rails

0 Answers