fix: lint errors (UP007, UP045, UP006 & UP035)

[bhimrazy]

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

This PR addresses lint errors across the codebase by:

• Removing unused lint ignores and cleaning up type annotations
• Fixing pyupgrade issues UP007 and UP045 (e.g., removing unnecessary imports and upgrading syntax)
• Resolving type issues related to multiprocessing Queue

follow-up to #751

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

❌ Patch coverage is 98.54369% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 80%. Comparing base (059b91f) to head (4506d63).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@         Coverage Diff         @@
##           main   #754   +/-   ##
===================================
  Coverage    80%    80%           
===================================
  Files        52     52           
  Lines      7366   7371    +5     
===================================
+ Hits       5907   5912    +5     
  Misses     1459   1459           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bhimrazy]

Checked

[Copilot]

Pull request overview

This PR modernizes type annotations across the codebase to comply with PEP 604 by converting from the older Optional[X] and Union[X, Y] syntax to the newer X | None and X | Y syntax. It also removes the corresponding pyupgrade lint rule ignores (UP007, UP045, UP006, UP035) from pyproject.toml.

Key Changes:

• Converted all Optional[X] annotations to X | None
• Converted most Union[X, Y] annotations to X | Y
• Removed unused imports of Optional and some Union from typing
• Updated pyproject.toml to remove UP007 and UP045 from lint ignore list
• Used string quotes for forward references involving Queue from multiprocessing to avoid circular import issues

Reviewed changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/streaming/test_dataset.py	Converted Optional to PEP 604 syntax in test fixtures
src/litdata/utilities/subsample.py	Updated type annotations for function parameters
src/litdata/utilities/parquet.py	Converted Union and Optional annotations across multiple classes
src/litdata/utilities/hf_dataset.py	Modernized function signatures with PEP 604 syntax
src/litdata/utilities/env.py	Updated Callable and Optional annotations, imported Callable from collections.abc
src/litdata/utilities/encryption.py	Converted Union annotations in state management methods
src/litdata/utilities/dataset_utilities.py	Updated Optional/Union annotations and moved Callable import
src/litdata/utilities/broadcast.py	Modernized HTTP client type annotations
src/litdata/utilities/base.py	Updated abstract base class type annotations
src/litdata/streaming/writer.py	Converted type annotations and used quoted "Queue" for forward reference
src/litdata/streaming/serializers.py	Updated serializer method signatures
src/litdata/streaming/sampler.py	Converted Iterator and Union annotations
src/litdata/streaming/resolver.py	Updated Dir-related type annotations (incomplete - see comments)
src/litdata/streaming/reader.py	Modernized reader class annotations
src/litdata/streaming/parallel.py	Updated dataset wrapper annotations
src/litdata/streaming/item_loader.py	Converted loader type annotations with quoted Queue reference
src/litdata/streaming/fs_provider.py	Updated filesystem provider signatures
src/litdata/streaming/downloader.py	Modernized downloader class annotations
src/litdata/streaming/dataset.py	Updated streaming dataset parameters (incomplete - see comments)
src/litdata/streaming/dataloader.py	Converted dataloader type annotations
src/litdata/streaming/config.py	Updated config class method signatures
src/litdata/streaming/combined.py	Modernized combined dataset annotations
src/litdata/streaming/client.py	Updated S3/R2 client type annotations
src/litdata/streaming/cache.py	Converted cache class annotations with quoted Queue
src/litdata/raw/indexer.py	Updated indexer abstract methods
src/litdata/raw/dataset.py	Modernized raw dataset type annotations
src/litdata/processing/utilities.py	Updated utility function signatures
src/litdata/processing/functions.py	Converted processing function annotations (incomplete - see comments)
src/litdata/processing/data_processor.py	Updated data processor type annotations
src/litdata/imports.py	Modernized requirement cache annotations
src/litdata/helpers.py	Updated helper function signatures
pyproject.toml	Removed UP007, UP045, UP006, UP035 from lint ignore list
examples/multi_modal/loop.py	Converted callback type annotations
examples/multi_modal/dataloader.py	Updated dataset class signatures
benchmarks/litdata/optimize_imagenet.py	Modernized resize_size annotation
.github/benchmark/benchmark.py	Updated benchmark configuration annotations

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.