Skip to content

Add native asyncio cursor support (Phase 1)#666

Merged
laughingman7743 merged 4 commits intomasterfrom
feature/native-asyncio-cursor
Feb 20, 2026
Merged

Add native asyncio cursor support (Phase 1)#666
laughingman7743 merged 4 commits intomasterfrom
feature/native-asyncio-cursor

Conversation

@laughingman7743
Copy link
Member

Summary

  • Add AioCursor and AioDictCursor that use native asyncio instead of ThreadPoolExecutor
  • Uses asyncio.sleep() for polling and asyncio.to_thread() for boto3 calls, keeping the event loop free
  • Add pyathena.aconnect() async entry point and AioConnection with async context manager
  • Add AioResultSet with async create() factory pattern (cannot await in __init__)
  • 16 integration tests passing

Motivation

Closes #662 (Phase 1). The existing AsyncCursor uses ThreadPoolExecutor which blocks threads during time.sleep() polling. Native asyncio support allows proper integration with async frameworks (FastAPI, aiohttp, etc.).

Scope (Phase 1 only)

This PR covers the core cursor types only:

  • AioCursor / AioDictCursor
  • AioResultSet / AioDictResultSet
  • AioBaseCursor / AioConnection

Deferred to Phase 2: Specialized cursors (pandas, arrow, polars, s3fs, spark) and metadata operations (list_databases, list_table_metadata, etc.).

Key design decisions

Decision Choice Rationale
Blocking calls asyncio.to_thread(retry_api_call, ...) Zero new deps, works with any boto3 version
Result set init async create() classmethod Cannot await in __init__
Naming Aio prefix (AioCursor) Async prefix already taken by thread-based cursor
Cancellation asyncio.CancelledError Correct async idiom (replaces KeyboardInterrupt)
Inheritance Extend sync BaseCursor Reuse _build_* methods, __init__, constants

Usage

import pyathena

conn = await pyathena.aconnect(
    s3_staging_dir="s3://bucket/path/",
    region_name="us-east-1",
)
async with conn.cursor() as cursor:
    await cursor.execute("SELECT * FROM my_table")
    rows = await cursor.fetchall()
    async for row in cursor:
        print(row)

Test plan

  • make fmt passes
  • make chk (ruff + mypy) passes
  • 16 async integration tests passing (tests/pyathena/aio/test_cursor.py)
  • CI pipeline passes

🤖 Generated with Claude Code

laughingman7743 and others added 4 commits February 20, 2026 20:24
Implement AioCursor and AioDictCursor that use asyncio.sleep() for
polling and asyncio.to_thread() for boto3 calls, keeping the event
loop free. This replaces the ThreadPoolExecutor-based approach used
by the existing AsyncCursor for users who need true asyncio support.

New files:
- pyathena/aio/ package (util, common, result_set, cursor, connection)
- tests/pyathena/aio/ (conftest, test_cursor with 16 tests)
- pyathena.aconnect() entry point

Closes #662 (Phase 1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rename AioResultSet -> AthenaAioResultSet, AioDictResultSet ->
  AthenaAioDictResultSet to match existing AthenaResultSet naming
- Use multiple inheritance (AthenaDictResultSet, AthenaAioResultSet) to
  eliminate duplicated _get_rows method in AthenaAioDictResultSet

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Google-style docstrings to AioCursor properties and methods
  matching the sync Cursor docstring patterns
- Add docstrings to AthenaAioResultSet fetch methods
- Fix _poll log message to match sync: "Query canceled by user."

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_description_initial: verify description is None before execution
- test_bad_query: verify DatabaseError on invalid SQL
- test_query_id: verify query_id is UUID v4 format
- test_query_execution_initial: verify all WithResultSet properties
  are in correct initial state before any query execution
- test_cancel_initial: verify ProgrammingError on cancel without query
- test_executemany: verify async executemany with INSERT
- test_executemany_fetch: verify result set cleared after executemany
- test_execute_with_callback: verify on_start_query_execution callback
- Add execute_many_aio table to test SQL template

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@laughingman7743 laughingman7743 marked this pull request as ready for review February 20, 2026 11:49
@laughingman7743 laughingman7743 merged commit f011ea8 into master Feb 20, 2026
5 checks passed
@laughingman7743 laughingman7743 deleted the feature/native-asyncio-cursor branch February 20, 2026 11:50
laughingman7743 added a commit that referenced this pull request Feb 21, 2026
Add comprehensive documentation for the native asyncio cursor
implementations added in PRs #666, #667, #668. This includes a new
docs/aio.md overview page, AioCursor sections in each specialized
cursor page, API reference for the aio module, and an async example
in the README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add native asyncio cursor support (AioCursor)

1 participant