Open
Conversation
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
…ne() logic - Fix AttributeError: replace sqlite_config.sqlite_path with rundb_sqlite_path in rundb.py and mongo_storage.py (3 locations) - Fix NameError in OfflineSQLiteCollection.find_one(): add proper _id handling and default query case - These bugs would cause immediate crashes when offline mode is activated Fixes ensure offline SQLite backend actually works when both rundb and xedocs SQLite files are present.
Tests cover: - SQLiteConfig loading and activation logic - OfflineGridFS file operations (list, download) - OfflineSQLiteCollection queries (find_one, find, count_documents) - xent_collection() fallback behavior (SQLite vs MongoDB) - Edge cases and error handling All 13 tests pass successfully.
- Added detailed section in README.md covering setup, usage, and troubleshooting - Documented environment variables and their purposes - Added examples for generating SQLite dumps and using offline mode - Included limitations and performance considerations - Enhanced module docstring in sqlite_backend.py with usage examples
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
) * Fix critical bugs: correct sqlite_path references and complete find_one() logic - Fix AttributeError: replace sqlite_config.sqlite_path with rundb_sqlite_path in rundb.py and mongo_storage.py (3 locations) - Fix NameError in OfflineSQLiteCollection.find_one(): add proper _id handling and default query case - These bugs would cause immediate crashes when offline mode is activated Fixes ensure offline SQLite backend actually works when both rundb and xedocs SQLite files are present. * Apply black formatting to sqlite_backend.py * Add comprehensive tests for offline SQLite backend Tests cover: - SQLiteConfig loading and activation logic - OfflineGridFS file operations (list, download) - OfflineSQLiteCollection queries (find_one, find, count_documents) - xent_collection() fallback behavior (SQLite vs MongoDB) - Edge cases and error handling All 13 tests pass successfully. * Add comprehensive documentation for offline SQLite backend - Added detailed section in README.md covering setup, usage, and troubleshooting - Documented environment variables and their purposes - Added examples for generating SQLite dumps and using offline mode - Included limitations and performance considerations - Enhanced module docstring in sqlite_backend.py with usage examples * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix pre-commit issues: syntax error and flake8 violations * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix flake8 issues in test file: line length and unused variables * Fix mypy type errors and flake8 whitespace issues * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix pre-existing flake8 and mypy issues in mongo_to_sqlite.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add noqa comments for unavoidable black/flake8 conflicts * Move noqa comment to correct line in test file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rewrite SQL as single-line strings to avoid black/flake8 conflict * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Allow long first line in SQL query with noqa * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove accidentally committed .DS_Store file * Add .DS_Store to gitignore --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
for more information, see https://pre-commit.ci
Member
Author
🎉 PR Ready for Review!This PR has been thoroughly reviewed and enhanced with: ✅ Critical Bug Fixes Applied
🧪 Comprehensive Test Suite Added
📚 Complete Documentation Added
🎨 Code Quality
The offline SQLite backend is now production-ready with proper testing and documentation! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces offline (SQLite-based) backend support to utilix, allowing core database and GridFS operations to work seamlessly when disconnected from the main MongoDB service. The main changes involve integrating the new
sqlite_backendand updating the relevant classes and methods to use it when configured.Offline backend integration:
sqlite_backendand its key classes/functions (OfflineGridFS,OfflineSQLiteCollection,SQLiteConfig,_load_sqlite_config) inutilix/__init__.py,utilix/mongo_storage.py, andutilix/rundb.pyto enable offline support throughout the package. [1] [2] [3]GridFS (file storage) offline support:
GridFsBaseclass inutilix/mongo_storage.pyto initialize and useOfflineGridFSwhen offline mode is detected, including offline-safe overrides for file listing, config existence checks, and file downloads. This ensures file storage and retrieval continue to work in offline scenarios. [1] [2]RunDB (database) offline support:
utilix/rundb.pyto provide an_sqlite_collectionfunction and updatexent_collectionto use the offline SQLite backend if enabled, allowing database queries to fallback to the local SQLite database when offline.General improvements and error handling:
Consistency and code reuse: