myTorch

A C++/CUDA neural network library inspired by PyTorch, built from scratch.

Features

• Tensors: CPU and GPU support with automatic memory management.
• Layers: Dense (Linear), ReLU, Softmax.
• Loss: Categorical Cross Entropy.
• Optimizer: Stochastic Gradient Descent (SGD).
• Backends: Custom CPU and CUDA kernels.

Requirements

• CMake >= 3.18
• CUDA Toolkit (for GPU support)
• C++17 Compiler

Build

mkdir build
cd build
cmake ..
make

Usage

Running the Example

The example trains a neural network on the "spiral data" problem.

CPU Mode:

./train_spiral

GPU Mode:

./train_spiral gpu

Library Structure

• include/: Header files.
• src/: Source code (C++ and CUDA).
• examples/: Example usage scripts.

Implementation Details

• Tensor: Handles data storage and device movement (to(Device::GPU)).
• Ops: Contains math operations (matmul, add, relu, etc.) with dispatch logic for CPU/GPU.
• Layers: High-level abstractions (Layer_Dense, Layer_ReLU) that store parameters and implement forward/backward passes.