Skip to content

Join operators don't yield to tokio runtime, preventing query cancellation #19358

@erratic-pattern

Description

@erratic-pattern

Describe the bug

Join operators (HashJoinExec, SortMergeJoinExec, CrossJoinExec, NestedLoopJoinExec) don't yield to the tokio runtime during execution, which prevents query cancellation from working. When a long-running join query is aborted, the cancellation is never processed because the operator never yields control back to tokio.

To Reproduce

Reproducer project: https://github.com/erratic-pattern/datafusion-join-cancellation-repro

Expected behavior

Join operators should periodically yield to the tokio runtime so that task cancellation requests are processed promptly and to avoid worker starvation for other tasks.

Additional context

A simple fix would be to wrap join streams with make_cooperative() in each operator's execute() method.

PR: #19360

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions