Skip to content

Conversation

@dgomesbr
Copy link
Contributor

@dgomesbr dgomesbr commented Jan 1, 2026

Feature/exponential backoff added to both Sonarr and Radarr clients with sensible defaults.

feat: implement batch processing to stabilize large library scans

- Add configurable batch processing (BATCH_SIZE, BATCH_DELAY_SECONDS)
- Process items in chunks to prevent API overwhelming and database locks
- Default to 100 items per batch with 10s delay between batches
- Maintain full backward compatibility with existing configurations
- Add smart messaging that only shows verbose output for large libraries
- Include comprehensive tests and documentation

Closes nullable-eth#26
Updating batch info on Readme
added batch information
@nullable-eth
Copy link
Owner

Hey @dgomesbr, thanks for these contributions! The batch processing and exponential backoff features look solid and address real issues for users with large libraries.

I found a couple of issues:

🔴 Bug: Missing time import in internal/config/config_test.go

The TestBackwardCompatibility function references time.Hour on line 185, but the time package isn't imported. Quick fix:

import (
	"os"
	"testing"
	"time"  // <- add this
)

⚠️ Potential Issue: Request body not re-readable on retry

In internal/utils/retry.go, the retry logic clones the request but doesn't handle request body re-reading:

reqClone := req.Clone(ctx)

If any request has a body (POST/PUT), the body stream will be consumed on the first attempt, and subsequent retries will send an empty body. While this may not be an issue currently since the Radarr/Sonarr clients only make GET requests, it could cause subtle bugs if POST/PUT requests are added later.

A safer approach would be to use GetBody if available, or buffer the body for retries:

if req.GetBody != nil {
    body, err := req.GetBody()
    if err == nil {
        reqClone.Body = body
    }
}

Not a blocker for this PR since the current usage is GET-only, but worth noting for future-proofing.


Also, since this PR contains all the changes from PR #27 plus the exponential backoff feature, I'll close #27 as superseded by this one.

Once the import fix is in, this should be good to merge! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants