git-los

A Git LFS custom transfer agent with native GCS and S3 support, daemon-based credential caching, and global installation.

Overview

git-los provides a seamless workflow for storing Git LFS objects in cloud object storage (Google Cloud Storage and Amazon S3) without requiring a dedicated LFS server.

Features

• Native cloud SDK support — First-class GCS and S3 backends using official SDKs
• Daemon architecture — Credential caching, connection pooling, background sync
• Global installation — Configure once, works across all repositories
• Standard LFS integration — Works with git lfs track, push, pull, etc.

Installation

Homebrew (macOS/Linux)

brew tap valent-au/tap
brew install git-los

From Source

go install github.com/valent-au/git-los/cmd/git-los@latest

See Installation Guide for more options including binary downloads.

Quick Start

# Install globally (configures git to use git-los for LFS transfers)
git-los install --global

# Configure a repository to use GCS
cd your-repo
git-los remote add origin gs://your-bucket/lfs

# Or use S3
git-los remote add origin s3://your-bucket/lfs

# Use git-lfs as normal
git lfs track "*.bin"
git add .gitattributes large-file.bin
git commit -m "Add large file"
git push  # LFS objects upload to your bucket via git-los

Prerequisites:

• git-lfs must be installed
• Cloud credentials configured (GCS: Application Default Credentials, S3: AWS credentials)

Documentation

User Guides

• Installation — Installing and setting up git-los
• Configuration — Backend setup and options
• Troubleshooting — Common issues and solutions

Example Workflows

• Game Development — Managing textures, models, and audio
• ML Datasets — Dataset versioning and model management

Development

• Contributing — How to contribute to git-los
• Development Guide — Building and testing locally

Specifications

See the spec directory for detailed specifications:

• Architecture — System components and their interactions
• Transfer Agent Protocol — Git LFS custom transfer agent implementation
• Daemon — Background daemon process
• Configuration — Config file formats and precedence
• Storage Backends — GCS and S3 backend implementations
• CLI — Command-line interface specification
• Security — Credential handling and security considerations

Development

Prerequisites

• Go 1.21 or later
• Docker (for integration tests)
• golangci-lint (for linting)

Pre-commit Hooks

Set up pre-commit hooks to run tests and linting before each commit:

# Create the pre-commit hook
cat > .git/hooks/pre-commit << 'EOF'
#!/bin/sh
set -e
go test ./...
go vet ./...
golangci-lint run
EOF
chmod +x .git/hooks/pre-commit

Building

go build ./cmd/git-los

Testing

# Run unit tests
make test

# Run with coverage
make coverage

# Run integration tests (requires Docker)
make test-integration

# Run linter
make lint

Project Structure

git-los/
├── cmd/git-los/         # Main binary
├── internal/
│   ├── agent/           # LFS protocol handling
│   ├── backend/         # Storage backends (GCS, S3)
│   ├── config/          # Configuration loading
│   ├── daemon/          # Background daemon
│   ├── ipc/             # Agent-daemon communication
│   └── testutil/        # Test helpers
├── spec/                # Specifications
└── tasks/               # Implementation tasks

License

See LICENSE for details.