Parallel File Processor Project Requirements.

Project Overview

Create a thread pool-based system for processing multiple files concurrently with configurable processing rules.

Core Requirements

Thread Pool Implementation

1. Must implement a generic thread pool
2. Must support dynamic number of worker threads
3. Must handle task distribution efficiently
4. Must implement proper shutdown mechanism

File Processing Features

1. Must support processing files from multiple directories
2. Must implement these analyzers:
   • Word count
   • Line count
   • Character frequency
   • File size statistics

Technical Requirements

1. Must use Arc and Mutex for shared state
2. Must implement proper error handling
3. Must support cancellation
4. Must handle all file system errors gracefully

Progress Tracking

1. Real-time progress updates
2. Per-file processing status
3. Error reporting with context
4. Processing time statistics

Output Format

struct FileAnalysis {
    filename: String,
    stats: FileStats,
    errors: Vec<ProcessingError>,
    processing_time: Duration,
}

struct FileStats {
    word_count: usize,
    line_count: usize,
    char_frequencies: HashMap<char, usize>,
    size_bytes: u64,
}

Bonus Features

1. Support for different file encodings
2. Extensible analyzer system
3. Memory usage limiting
4. Progress persistence
5. Resume capability

Testing Requirements

1. Unit tests for thread pool
2. Integration tests for file processing
3. Performance benchmarks
4. Error handling scenarios