Image Processing Benchmark ⚡📊

Concurrency Wars. A technical showdown between Python and Golang architectures in high-demand batch processing scenarios.

💡 About the Project

Image Processing Benchmark was developed as a technical validation study for a larger Bachelor's Thesis. Its mission is to demonstrate the practical implications of different concurrency models and library optimizations when handling CPU-bound tasks.

The project compares standard approaches in Python (Threading, Multiprocessing) against Golang (Native & C++ Bindings). The goal is to process a batch of 1,000 high-resolution images (4K) to measure Execution Time, CPU Utilization, and Memory Footprint, providing data-driven justification for architectural choices in scalable software.

🛠️ Tech Stack

🧩 Tested Architectures

1. Python Multithreading

• Definition: Uses standard OS threads managed by Python (concurrent.futures.ThreadPoolExecutor).
• The Catch: Due to the GIL (Global Interpreter Lock), Python cannot execute bytecode in parallel.
• Why it works here: Libraries like OpenCV (C++) release the GIL during heavy calculations, allowing partial parallelism.

2. Python Multiprocessing

• Definition: Bypasses the GIL by spawning separate Operating System Processes. Each worker has its own independent memory space.
• The Catch: Memory duplication. Creating 12 processes means loading the Python runtime and libraries 12 times, leading to exponential RAM usage.

3. Golang Native

• Definition: Uses Go's standard library (image/draw) and Goroutines.
• The Catch: While Goroutines are efficient, the standard Go library processes pixels on the CPU without SIMD/AVX hardware optimizations (which C++ libraries use by default).

4. Golang + GoCV

• Definition: Uses Goroutines for lightweight orchestration but delegates the heavy math to OpenCV (C++) via bindings.
• The Win: Achieves the raw speed of C++ while maintaining the ultra-low orchestration overhead of Go.

---

🖥️ Test Environment

All benchmarks were executed on a controlled local environment:

Component	Specification
CPU	AMD Ryzen 5 5500 (6 Cores / 12 Threads @ 3.6-4.2 GHz)
RAM	32GB (2x16GB) DDR4 3200MHz CL16
GPU	NVIDIA RTX 2060 Super (8GB)
OS	CachyOS (Arch Linux based)

📈 Benchmark Results

Tests performed processing a batch of 1,000 High-Resolution Images (4K).

Architecture	Execution Time	Peak Memory (RAM)	Efficiency Verdict
Python (Multithreading)	49.56 s	938.27 MB	Low (GIL Bottleneck)
Python (Multiprocessing)	43.54 s	2,041.40 MB	Heavy (High RAM Cost)
Golang (Native Lib)	2m 24s	5,344.95 MB	Slow (Lack of SIMD)
Golang + GoCV (Hybrid)	11.76 s 🚀	1,575.48 MB	Ultra Efficiency

📊 Analysis:

• Speed: The Go + GoCV approach was 3.7x faster than the best Python attempt. This proves that Go's scheduler handles concurrent tasks significantly better than Python's multiprocessing engine.
• Memory: Python Multiprocessing consumed over 2GB to process the batch 4x slower. Go + GoCV used ~1.5GB to process it 4x faster. The efficiency ratio (Throughput per MB) of Go is vastly superior.

🏆 Why Golang? (Conclusion)

The core hypothesis of this benchmark was that Golang offers the best balance between performance and resource efficiency. The results confirm this:

1. Best Orchestration: Spawning 1,000 tasks in Go is trivial and virtually free (2KB per Goroutine). In Python, managing processes creates significant overhead.
2. The Best of Both Worlds: By using Go bindings for C++ libraries (GoCV), we access hardware-accelerated speeds that pure Python or pure Go cannot match alone.
3. Scalability: In a production environment (e.g., a Kubernetes Pod with limited RAM), the Python Multiprocessing approach would likely crash (OOM) under load, while the Go architecture remains stable.

---

🚀 How to Run

1. Clone the repository

   git clone https://github.com/nathanhgo/image-processing-benchmark.git
   cd image-processing-benchmark

2. Setup Dependencies

• Python:

      pip install opencv-python psutil    

• Golang + OpenCV: You need OpenCV installed on your system to run the hybrid benchmark.

  # Arch Linux based distros
  sudo pacman -S opencv vtk glew hdf5
  
  # Ubuntu based distros
  sudo apt install libopencv-dev libvtk9-dev libglew-dev libhdf5-dev
  
  # Windows (It is recommended to use the official GoCV build script)
  # Inside your GOPATH/src/gocv.io/x/gocv
  win_build_opencv.cmd
  
  # Initialize Go Modules
  go mod init benchmark-tcc
  go mod tidy

3. Prepare Test Data Place a high-resolution image named test.jpg inside the assets/ folder.

4. Run Benchmarks

   # 1. Run Python Multithreading
   python python/bench_thread.py
   
   # 2. Run Python Multiprocessing
   python python/bench_process.py
   
   # 3. Run Golang Native
   go run go/native/main.go
   
   # 4. Run Golang + GoCV (The Winner)
   go run go/gocv/main.go

---

Developed by: @nathanhgo