Fail loudly if a job is pushed to an undeclared queue

[dalberto]

Problem

Currently, if a job is pushed to a queue that has not been declared in Chancy, the operation appears to succeed silently. The job is inserted into the database and remains there, but no workers will ever process it, leading to silent failure and confusion for users.

Desired behavior

It should be possible for Chancy to fail loudly (raise an error, log visibly, etc.) if a job is pushed to a queue that has not first been declared. Ideally, this would be either the default behavior or configurable (e.g., a strict_queues option on the Chancy instance or on push).

Rationale

• Prevents silent job loss/confusion
• Easier debugging for developers and users
• More robust and predictable system behavior

Example

# This should raise an error if 'undeclared_queue' was not declared
await chancy.push(my_job.job.with_queue("undeclared_queue"))

Currently, I'm using this as a workaround:

from chancy import Chancy, Job, Queue

class StrictChancy(Chancy):
    async def push(self, job):
        # Get the queue name from the job
        queue_name = job.queue if isinstance(job, Job) else job.job. queue
        
        # Check if queue exists
        queue = await self.get_queue(queue_name)
        if queue is None:
            raise ValueError(f"Queue '{queue_name}' has not been declared. Declare it first)")
        
        return await super().push(job)

[TkTech]

The lack of referential integrity on the queue names was by design and significantly increased the speed of bulk insertions (in the tens of millions, see https://github.com/TkTech/chancy/blob/main/tests/test_scale.py). Since we can dynamically create and destroy queues pre-seeding them before announcing the queue is a valid pattern.

At the same time I can see this being useful. At a minimum you'd want this in push_many_ex instead of push, collecting all the unique queues in one pass and checking them in one go instead of for each and every job. Even better would be to push it down to the query.