feat: Add mssql-python driver support with pyodbc fallback#350
Open
dlevy-msft-sql wants to merge 3 commits intomicrosoft:mainfrom
Open
feat: Add mssql-python driver support with pyodbc fallback#350dlevy-msft-sql wants to merge 3 commits intomicrosoft:mainfrom
dlevy-msft-sql wants to merge 3 commits intomicrosoft:mainfrom
Conversation
9e70310 to
746e0d8
Compare
This PR introduces Microsoft's native mssql-python driver as the default database driver for Python 3.10+, while maintaining pyodbc as a fallback for Python 3.9 users and specific edge cases. Key changes: - New driver_backend abstraction layer (DriverBackend ABC) - MssqlPythonBackend and PyodbcBackend implementations - Auto-detection with fallback: mssql-python -> pyodbc - Environment variable override: DBT_FABRIC_DRIVER_BACKEND - Comprehensive documentation and migration guide - Extended unit test coverage (159 tests, 51% coverage) - Updated CI matrix for both drivers across Python 3.9-3.13
- Add _backend_lock for thread-safe backend initialization - Fix caching logic to track preference string, not resolved name - Add DEFAULT_BACKEND and VALID_BACKENDS constants - Consolidate UID/PWD handling to reduce duplication - Add InterfaceError to PyodbcBackend retryable exceptions - Remove redundant InterfaceError logic from connection manager - Fix Union[str, str] type hint and remove unused import - Remove hardcoded pooling in PyodbcBackend.connect()
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces Microsoft's native
mssql-pythondriver as the default database driver for dbt-fabric, replacingpyodbcas the primary option while maintaining full backward compatibility.Why this change?
pip installThe mssql-python driver bundles the native SQL Server driver, eliminating the need for users to install and configure ODBC drivers separately.
Changes
New Files
dbt/adapters/fabric/driver_backend.pyDriverBackendABC,MssqlPythonBackend, andPyodbcBackendimplementationstests/unit/adapters/fabric/test_driver_backend.pytests/unit/adapters/fabric/test_fabric_credentials.pytests/unit/adapters/fabric/test_fabric_connection_manager.pytests/unit/adapters/fabric/test_fabric_column.pytests/unit/adapters/fabric/test_fabric_relation.pyModified Files
fabric_credentials.pydriver_backendfield (auto/mssql-python/pyodbc)fabric_connection_manager.pyDriverBackendinterfacesetup.pyREADME.mdCHANGELOG.mdunit-tests.ymlintegration-tests-azure.ymltests/conftest.pydriver_backendto test profileDriver Selection Behavior
Configuration Examples
Automatic (Recommended)
Force Specific Driver
Environment Variable Override
Test Coverage
driver_backend.pyfabric_credentials.pyfabric_column.pyfabric_relation.pyfabric_connection_manager.pyTest count: 30 → 159 tests (+129 new tests)
Breaking Changes
None. This is a backward-compatible change:
pyodbcusers can continue usingdriver_backend: pyodbcdriverfield is still supported for pyodbc backendDeprecation Notice
The
driverfield will emit a warning when usingmssql-pythonbackend (where it's ignored).CI/CD Updates
Unit Tests Matrix
Integration Tests
Both drivers tested against live Fabric DW with Azure AD authentication.
System Dependencies
mssql-python (bundled driver)
brew install opensslapt install libltdl7 libkrb5-3 libgssapi-krb5-2dnf install libtool-ltdl krb5-libszypper install libltdl7 libkrb5-3 libgssapi-krb5-2apk add libtool krb5-libs krb5-devpyodbc (external driver)
Requires Microsoft ODBC Driver 18 for SQL Server installation.
Checklist