Workers keep trying more than 3 attempts

[davidakachaos]

Transferred from: davidakachaos/workless_revived#13

In the scaler options, in the docs, you say:

There are three other scalers included. Note that if you are running on the Aspen or Bamboo stacks on Heroku and you don't explicitly specify the scaler, the heroku scaler will be used automatically.

Delayed::Job.scaler = :heroku
Delayed::Job.scaler = :heroku
Delayed::Job.scaler = :local

Where you modified original documentation, where :heroku_cedar was an option. Then you say:

The heroku scaler works on the Aspen and Bamboo stacks while the heroku_cedar scaler only works on the new Cedar stack.

I've tried to push with old :heroku_cedar but I get from Heroku:

NameError: uninitialized constant Delayed::Workless::Scaler::HerokuCedar

Is that Cedar name related to the Heroku Stack?

I have the gem working now with :heroku scaler, but I've noticed issues with the dynos not shutting down. I think this can be related to it. Any idea?

[unmultimedio]

Brought also from davidakachaos/workless_revived#13

After some more debugging, I've found that the worker just kept trying to complete the job over and over again, and it was already the 11th try.

In Failing Jobs description says that workless try just 3 times and then removes the job. Seems that that setting isn't working 😕

Here's an extract of the logs from Heroku logs:

Jun 29 17:38:29 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (0 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:38:29 my-heroku-app-name app/worker.1: E, [2017-06-29T22:38:29.484937 #4] ERROR -- : 2017-06-29T22:38:29+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (0 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:38:36 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (1 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:38:36 my-heroku-app-name app/worker.1: E, [2017-06-29T22:38:36.145399 #4] ERROR -- : 2017-06-29T22:38:36+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (1 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:39:02 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (2 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:39:02 my-heroku-app-name app/worker.1: E, [2017-06-29T22:39:01.177675 #4] ERROR -- : 2017-06-29T22:39:01+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (2 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:40:31 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (3 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:40:31 my-heroku-app-name app/worker.1: E, [2017-06-29T22:40:31.246339 #4] ERROR -- : 2017-06-29T22:40:31+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (3 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:44:56 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (4 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:44:56 my-heroku-app-name app/worker.1: E, [2017-06-29T22:44:56.453852 #4] ERROR -- : 2017-06-29T22:44:56+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (4 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:55:27 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (5 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 17:55:27 my-heroku-app-name app/worker.1: E, [2017-06-29T22:55:27.266383 #4] ERROR -- : 2017-06-29T22:55:27+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (5 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 18:17:10 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (6 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 18:17:10 my-heroku-app-name app/worker.1: E, [2017-06-29T23:17:10.322653 #4] ERROR -- : 2017-06-29T23:17:10+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (6 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 18:57:20 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (7 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 18:57:20 my-heroku-app-name app/worker.1: E, [2017-06-29T23:57:19.552528 #4] ERROR -- : 2017-06-29T23:57:19+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (7 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 20:05:45 my-heroku-app-name app/worker.1: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (8 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29 
Jun 29 20:05:45 my-heroku-app-name app/worker.1: E, [2017-06-30T01:05:44.591284 #4] ERROR -- : 2017-06-30T01:05:44+0000: [Worker(host:random-id-of-the-host pid:4)] Job ActiveJob::QueueAdapters::DelayedJobAdapter::JobWrapper (id=5109) FAILED (8 prior attempts) with ActiveJob::DeserializationError: Error while trying to deserialize arguments: Couldn't find Invitation with 'id'=29

[patricklindsay]

Yes that setting has no effect. You'll have to set the max_attempts yourself.

[unmultimedio]

Thanks @patricklindsay, how do you do that? In the initializer to override this? https://github.com/lostboy/workless/blob/master/lib/workless/initialize.rb#L3

[patricklindsay]

Create an initialiser within your application and set it there. For example;

# config/initializers/delayed_job.rb

Delayed::Worker.max_attempts = 3

The reason why the setting isn't working in this repo is because it is doing an or equals `||=. This will only set the variable if it doesn't already have a value. In this case it has a value of 25 which is the default set by Delayed Jobs.