Provide configuration to override using Scheduler while re-queuing even if its present #12
Description
checks for presence of Resque::Scheduler(http://github.com/resque/resque-scheduler). If it is present, jobs do not get re-enqueued to queue directly rather they always get offloaded to delayed_queue. This behavior may be not a required behavior always(see use case below). Is is possible to provide an env var to override this and reenqueue to queue instead of using scheduler even its present? A workaround we tried was to just override reenqueue method in our job code.
Use Case: We have around 10-20 jobs which used to have same lock, and queue size of at least 300K such jobs, in this scenario a lot jobs ended up in delayed queues, The delayed queue processing is done by only one master process and we observed the scheduler is not dispatching the jobs to main queue fast enough(job timestamps in delayed queue older than Time.now) due to volume of delayed jobs. This volume of delayed jobs due to worker locks affected the dispatching of the resque-retry jobs which were in delayed queue due to failures. This degraded our job processing SLAs. We were able to mitigate the issue by overriding reenqueue, however the behavior was not cleat until we looked in to code of resque-workers-lock. It would help if you can add some doc in context of behavior in presence of resque scheduler.
I would be interested in knowing your thoughts on the above use case. I would happy to raise PRs for the change request and documentation change if required.