fix #173

[askobara]

What about the solution such this? It's not finished yet, just to pic the idea of using yii/redis/Mutex to solve the concurrency issue.

Q	A
Is bugfix?	yes
New feature?	no
Breaks BC?	no
Tests pass?	yes
Fixed issues	#173

[samdark]

The idea itself sounds good to me.

[samdark]

That cold lead to fail-add indefinite cycle. Should be limited number of re-adding the job back.

[askobara]

Yep, you are right, but I rely on a class/global defined number of attempts. In this case if all attempts have been used up, job would be marked as a handled and this condition will never be invoked.

[samdark]

Aha.

[samdark]

@zhuravljov what do you think?

[askobara]

Also, a few things are confusing me a bit in this PR:

1. Method reserve doesn't make a reserve actually.
2. Placing a job's id into "reserved" queue are making from two parts of the class.

Should I fix it?

[askobara]

Any updates?
I'm already using this patch in production, and seems like it's works.

[rob006]

https://github.com/yiisoft/yii2-redis/blob/master/src/Mutex.php#L113-L119

If lock cannot be set, redis mutex will try again after 1 second. 1 second is like ages in this context - I'm able to handle tens of jobs in this time.

[askobara]

Implementation of the mutex can be changed via config. It's only default behavior.

[rob006]

Right now instance of yii\redis\Mutex is required. Also remove() and clear() uses higher frequency for mutex check - why reserving job should be slower?

[askobara]

Yeah, right now yii\redis\Mutex is using, but no one forbid to inherit some class from it and use this implementation in the queue. Yii provides yii\redis\Mutex with 1 second delay, by default. I guess, this issue is not about the PR.

However, you are right about the unfairly low delay of other actions. I could suggest to try to use same mutex for those actions and also, start a discussion in the main repo about decreasing delay of yii\redis\Mutex.

What do you think?

[rob006]

Yeah, right now yii\redis\Mutex is using, but no one forbid to inherit some class from it and use this implementation in the queue.

But there is no valid reason for requiring yii\redis\Mutex, any mutex component will work fine.

[askobara]

Oh, I get it.

[cebe]

https://github.com/yiisoft/yii2-redis/blob/master/src/Mutex.php#L113-L119

If lock cannot be set, redis mutex will try again after 1 second. 1 second is like ages in this context - I'm able to handle tens of jobs in this time.

would be a good idea to make the timeout a float in redis mutex to allow retry much faster.