-
Notifications
You must be signed in to change notification settings - Fork 107
Closed
Description
Problem
The five sync cursor implementations (Cursor, PandasCursor, ArrowCursor, PolarsCursor, S3FSCursor) duplicate ~350 lines of identical boilerplate code:
| Duplicated code | Lines (×5 cursors) |
|---|---|
Properties: result_set, query_id, rownumber, rowcount |
~125 |
Fetch methods: fetchone, fetchmany, fetchall |
~75 |
Lifecycle: close(), cancel(), executemany() |
~60 |
execute() shared control flow |
~100 |
This is the sync equivalent of the deduplication done for async cursors in #668 (WithAsyncFetch mixin).
Proposed solution
Create a WithFetch mixin in pyathena/common.py (mirroring WithAsyncFetch in pyathena/aio/common.py) that provides:
- Properties:
arraysize(getter/setter),result_set(getter/setter),query_id(getter/setter),rownumber,rowcount - Lifecycle:
close(),executemany(),cancel() - Default fetch:
fetchone(),fetchmany(),fetchall()— delegates toself.result_set.fetch*()
Each specialized cursor would then only need to implement __init__, get_default_converter(), execute(), and format-specific methods (e.g., as_pandas(), as_arrow()).
Files involved
| File | Change |
|---|---|
pyathena/common.py |
Add WithFetch(BaseCursor, CursorIterator, WithResultSet) |
pyathena/cursor.py |
Extend WithFetch, remove boilerplate |
pyathena/pandas/cursor.py |
Extend WithFetch, remove boilerplate |
pyathena/arrow/cursor.py |
Extend WithFetch, remove boilerplate |
pyathena/polars/cursor.py |
Extend WithFetch, remove boilerplate |
pyathena/s3fs/cursor.py |
Extend WithFetch, remove boilerplate |
Context
- This was identified during Phase 3 async cursor work (Add native asyncio S3FS and Spark cursors with boilerplate deduplication (Phase 3) #668)
- The async side already has
WithAsyncFetchinpyathena/aio/common.pyas a working reference S3FSCursorwill need async fetch overrides (same pattern asAioS3FSCursor)SparkCursoris excluded (different execution model, no result set)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels