-
Notifications
You must be signed in to change notification settings - Fork 1
Parallelize metadata scanning #89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Release Notes WIP 🚀 New: Parallel Metadata Scans + Better Job ErrorsThis release introduces a new parallel metadata scanner and much better error visibility for both scan and thumbnail jobs. 🧠 New parallel metadata writer for SCAN jobs
You can see these details directly in the Job History UI.
This gives you control over performance vs. resource usage, instead of baking in a fixed parallelism strategy. 🖼 Thumbnail job error details
📊 Job History UI upgrades
Overall, scans should be faster on larger libraries, and when something goes wrong, you’ll have much clearer insight into exactly what and why. |
Summary
This PR introduces a new parallel metadata writer for SCAN jobs and adds rich error reporting for both SCAN and THUMBNAIL pipelines, along with UI support in the admin Job History page.
The goals are:
Key Changes
🧠 New parallel metadata writer (SCAN)
Implemented a dedicated metadata writer process that:
Introduced a structured
error_detailslist for SCAN jobs, containing per-file failures:file_pathmessageExample SCAN job summary:
{ "imported": 10, "updated": 5, "errors": 2, "skipped": 0, "error_details": [ { "file_path": "/path/to/bad.cbz", "message": "Missing ComicInfo.xml" }, { "file_path": "/path/to/other.cbz", "message": "Failed to parse metadata" } ] }