feat: implement maxdepth and withdirs options for S3FileSystem.find()

[laughingman7743]

Summary

This PR implements the maxdepth and withdirs options for the S3FileSystem.find() method, providing compatibility with s3fs behavior.

Changes

• ✅ maxdepth parameter: Limits directory traversal depth using an efficient recursive approach
• ✅ withdirs parameter: Controls whether directories are included in results (default: False)
• ✅ Improved cache management: Uses (path, delimiter) tuple as cache key
• ✅ Code refactoring: Extracted directory generation logic to _extract_parent_directories() method
• ✅ Comprehensive tests: Added tests for both parameters
• ✅ Documentation: Updated CLAUDE.md with development guidelines

Implementation Details

maxdepth

• Uses recursive traversal with delimiter="/" for efficiency (similar to s3fs)
• Only fetches directories up to the specified depth, avoiding unnecessary S3 API calls
• maxdepth=0: Returns only files directly in the specified path
• maxdepth=1: Returns files in the path and one level of subdirectories

withdirs

• Default is False (returns only files, consistent with typical user expectations)
• When True, includes directory entries in the results
• For unlimited depth queries, directories are derived from file paths

Code Quality Improvements

• Extracted _extract_parent_directories() method to reduce _find() method complexity
• Added Google-style docstring with clear parameter and return value documentation
• Follows the principle of single responsibility for better maintainability

Performance Improvements

• When maxdepth is specified, uses level-by-level traversal instead of fetching all files
• Significantly faster for shallow searches in large directory trees
• Reduces S3 API calls and data transfer

Cache Management

• Replaced the initial approach of adding delimiter field to S3Object
• Now uses (path, delimiter) as the cache key
• Ensures different delimiter queries don't conflict in the cache

Testing

All tests pass successfully:

• test_find_maxdepth: Verifies correct depth limiting at multiple levels
• test_find_withdirs: Verifies directory inclusion/exclusion behavior
• All existing tests continue to pass

Documentation Updates

Updated CLAUDE.md with:

• Google-style docstring guidelines
• Testing best practices and examples
• S3 FileSystem development guidelines
• Performance optimization tips

Compatibility

The implementation follows s3fs conventions for both parameters, ensuring drop-in compatibility for users migrating from s3fs.

🤖 Generated with Claude Code

[laughingman7743]

TODO:

• Since the list_objects_v2 results differ depending on the delimiter, it is necessary to consider how to keep the differences in the cache.