Nexus - Next-Generation Distributed Version Control System

A modern distributed version control system designed for monorepos, massive binaries, and AI model versioning.

🚀 Features

• Content-Addressable Storage: BLAKE3-based hashing with chunk-level deduplication
• CRDT-Based Merging: Conflict-free merges using Automerge for automatic conflict resolution
• Partial Clones: Smart filtering with semantic queries (e.g., "only final models", "checkpoints > 90% accuracy")
• AI Model Versioning: Native support for tracking experiments, hyperparameters, metrics, and model lineage
• Large Binary Support: Efficient chunking and compression for multi-GB files
• Semantic History: Query models by metrics, experiments, and lineage

🎯 Target Users

• AI/ML Companies: Version control for models, datasets, and experiments
• Game Studios: Manage large binary assets (textures, models, audio)
• Infrastructure Teams: Handle monorepos with mixed content types

📦 Installation

From Source

git clone https://github.com/bhaskarvilles/dvc.git
cd dvc
cargo build --release
cargo install --path .

🔧 Quick Start

Initialize a Repository

nexus init my-project
cd my-project

Add and Commit Files

nexus add .
nexus commit -m "Initial commit"

Create Branches

nexus branch feature/new-model

Partial Clone

Clone only specific files or models:

# Clone only Python files
nexus clone --partial --filter "*.py" https://example.com/repo.git

# Clone only final models with accuracy > 0.9
nexus clone --partial --filter "semantic:metric_threshold:accuracy:0.9" https://example.com/repo.git

AI Model Versioning

Track model metadata in commits:

# Commit with model metadata
nexus commit -m "Trained ResNet50" \
  --metadata model_name=resnet50 \
  --metadata accuracy=0.95 \
  --metadata framework=pytorch

📚 Documentation

Core Concepts

Content-Addressable Storage

All objects are stored using BLAKE3 hashing, ensuring:

• Deduplication: Identical content stored once
• Integrity: Content verified on retrieval
• Efficiency: Chunk-level deduplication for large files

CRDT Merges

Nexus uses Conflict-free Replicated Data Types (CRDTs) for automatic merge resolution:

• Text files: Operational transformation
• JSON files: Map-based CRDTs
• Binary files: Three-way merge fallback

Semantic History

Track AI models with rich metadata:

• Hyperparameters
• Training metrics
• Dataset information
• Model lineage (fine-tuning chains)
• Experiment grouping

Commands

# Repository management
nexus init [path]              # Initialize repository
nexus status                   # Show working directory status
nexus log [-n count]           # Show commit history

# Version control
nexus add <files>              # Stage files
nexus commit -m "message"      # Create commit
nexus branch [name]            # Create/list branches
nexus merge <branch>           # Merge branches

# Remote operations
nexus clone <url> [path]       # Clone repository
nexus push [remote] [branch]   # Push changes
nexus pull [remote] [branch]   # Pull changes

🏗️ Architecture

┌─────────────────────────────────────────┐
│           CLI Interface                 │
└─────────────┬───────────────────────────┘
              │
┌─────────────▼───────────────────────────┐
│       Repository Manager                │
├─────────────┬───────────────────────────┤
│  ┌──────────▼──────────┐                │
│  │ Content-Addressable │                │
│  │     Storage (CAS)   │                │
│  ├─────────────────────┤                │
│  │  • BLAKE3 Hashing   │                │
│  │  • Compression      │                │
│  │  • Deduplication    │                │
│  └─────────────────────┘                │
│                                          │
│  ┌──────────────────────┐               │
│  │   CRDT Merge Engine  │               │
│  ├──────────────────────┤               │
│  │  • Text Merging      │               │
│  │  • JSON Merging      │               │
│  │  • Binary Fallback   │               │
│  └──────────────────────┘               │
│                                          │
│  ┌──────────────────────┐               │
│  │  Partial Clone       │               │
│  ├──────────────────────┤               │
│  │  • Path Filters      │               │
│  │  • Size Filters      │               │
│  │  • Semantic Filters  │               │
│  └──────────────────────┘               │
│                                          │
│  ┌──────────────────────┐               │
│  │  Semantic History    │               │
│  ├──────────────────────┤               │
│  │  • Model Metadata    │               │
│  │  • Experiment Track  │               │
│  │  • Lineage Graph     │               │
│  └──────────────────────┘               │
└──────────────────────────────────────────┘

🧪 Development

Build

cargo build

Run Tests

cargo test

Run Benchmarks

cargo bench

Code Coverage

cargo tarpaulin --out Html

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

📄 License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option.

🙏 Acknowledgments

• Automerge for CRDT implementation
• BLAKE3 for fast hashing
• Git for inspiration and design patterns

🗺️ Roadmap

• Core version control primitives
• Content-addressable storage
• CRDT-based merging
• Partial clone support
• Semantic history for AI models
• Real-time collaboration (Phase 2)
• WebSocket-based sync
• Distributed garbage collection
• Performance optimizations for 100GB+ repos

📧 Contact

For questions or support, please open an issue on GitHub.