Skip to content

Conversation

@breenyoung
Copy link
Collaborator

@breenyoung breenyoung commented Dec 29, 2025

Summary

This PR introduces a new parallel metadata writer for SCAN jobs and adds rich error reporting for both SCAN and THUMBNAIL pipelines, along with UI support in the admin Job History page.

The goals are:

  • Speed up library scans with a proper parallel metadata pipeline.
  • Make it clear which files/comics failed and why.
  • Let admins choose whether to use parallel scanning and how many workers to run.
  • Surface that information directly in the UI instead of forcing people into logs.

Key Changes

🧠 New parallel metadata writer (SCAN)

  • Implemented a dedicated metadata writer process that:

    • Receives worker results via a queue.
    • Applies metadata updates in batches using a single DB session.
    • Mirrors the behavior and invariants of the original synchronous scanner as closely as possible.
  • Introduced a structured error_details list for SCAN jobs, containing per-file failures:

    • file_path
    • message

Example SCAN job summary:

{
  "imported": 10,
  "updated": 5,
  "errors": 2,
  "skipped": 0,
  "error_details": [
    { "file_path": "/path/to/bad.cbz", "message": "Missing ComicInfo.xml" },
    { "file_path": "/path/to/other.cbz", "message": "Failed to parse metadata" }
  ]
}

@breenyoung
Copy link
Collaborator Author

breenyoung commented Dec 29, 2025

Release Notes WIP

🚀 New: Parallel Metadata Scans + Better Job Errors

This release introduces a new parallel metadata scanner and much better error visibility for both scan and thumbnail jobs.

🧠 New parallel metadata writer for SCAN jobs

  • Metadata scanning now uses a dedicated writer process to apply changes in batches.
  • This speeds up scans on larger libraries while keeping behavior aligned with the previous single-process scanner.
  • SCAN jobs now include detailed error_details in their summary:
    • Which files failed.
    • Why they failed (e.g., missing ComicInfo.xml, parse errors).

You can see these details directly in the Job History UI.

  • New admin settings:
    • Enable parallel metadata scanning
      • Turn the parallel path on or off.
      • Leave it off if you prefer the simpler, single-process behavior.
    • Parallel metadata workers
      • Set a specific number of worker processes.
      • Or leave it at 0/empty to let Parker choose a safe value based on your CPU cores.

This gives you control over performance vs. resource usage, instead of baking in a fixed parallelism strategy.

🖼 Thumbnail job error details

  • Thumbnail generation now reports per-comic errors instead of just an error count.
  • For each failure you’ll see:
    • Comic ID.
    • File path.
    • A plain-text error message (e.g., image load failures, unsupported format).

📊 Job History UI upgrades

  • The job details modal now displays error details for both SCAN and THUMBNAIL jobs.
  • The layout adapts based on job type:
    • SCAN jobs → compact table view (easy to skim many file-level errors).
    • THUMBNAIL jobs → detailed cards (more context per comic).

Overall, scans should be faster on larger libraries, and when something goes wrong, you’ll have much clearer insight into exactly what and why.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants