CommitLens

AI-powered terminal tool that critiques commit message quality and helps you write clear, high-signal commits from your shell.

CommitLens analyzes Git history, scores message quality, and suggests well-structured Conventional Commit messages from staged changes.

---

✨ Why this exists

Commit messages are often:

• vague (fixed bug)
• noisy (wip)
• missing context
• inconsistent across teams

CommitLens turns commit history into actionable feedback and helps teams write clearer commits consistently.

---

🧠 Features

Analysis mode

• Analyze the last N commits from local repositories
• Optionally analyze public remote repositories via --url
• AI critique + score (0-10)
• Suggestions for weak commits
• Stats dashboard (average score, vague %, one-word %)

Interactive mode

• Reads git diff --staged
• Summarizes staged changes
• Suggests a Conventional Commit message
• You always review/edit manually (tool never runs git commit)

Engineering quality

• Structured LLM output validation with Pydantic
• Rich terminal UX with progress and panels
• Diff filtering for lockfiles/binary assets
• Large diff truncation for prompt safety
• Lightweight eval harness for scoring behavior
• Minimal test suite for parsing/scoring/git-validation logic

---

🛠 Tech stack

• Python 3.11+
• OpenAI API
• Pydantic
• Rich
• Typer
• python-dotenv
• uv

---

✅ Prerequisites

• Python 3.11+
• Git
• Python package manager (recommended: uv, install with pip install uv)
• OpenAI API key

---

⚡ Quick start

git clone <your-repo-url>
cd commitlens
cp .env.example .env
# add OPENAI_API_KEY to .env
uv sync
uv run python commit_critic.py --analyze --limit 10

---

🚀 Installation

With uv (recommended)

uv sync

Without uv

python -m venv .venv && source .venv/bin/activate && pip install .

Create .env:

OPENAI_API_KEY=your_key_here

---

🔐 Security

• OPENAI_API_KEY is read from environment variables (or local .env) at runtime.
• CommitLens does not write your API key to project files.
• .env is git-ignored; only .env.example is tracked.
• If a key is exposed, rotate it immediately in your OpenAI dashboard.
• Use least-privilege practices: keep keys local, do not paste keys into commit messages, issues, or logs.

---

📦 Usage

# Analyze last 50 commits (local repo)
uv run python commit_critic.py --analyze

# Analyze last 50 commits from a remote public repo
uv run python commit_critic.py --analyze --url="https://github.com/steel-dev/steel-browser"

# Interactive commit writer
uv run python commit_critic.py --write

Run without uv

source .venv/bin/activate
python commit_critic.py --analyze
python commit_critic.py --analyze --url="https://github.com/steel-dev/steel-browser"
python commit_critic.py --write

Help

uv run python commit_critic.py --help

---

📊 Example output

Analyze mode (real sample)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━
💩 COMMITS THAT NEED WORK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────────────────────────────────────────────────────────────────────────╮
│ Commit: "add github action and quick start guide"                           │
│ Score: 3/10                                                                 │
│ Issue: Missing type prefix and scope; message is vague and not capitalized. │
│ Better: ci: add GitHub Action and docs: add quick start guide               │
╰─────────────────────────────────────────────────────────────────────────────╯

━━━━━━━━━━━━━━━━━━━━━━━━━━━━
💎 WELL-WRITTEN COMMITS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ Commit: "chore: add MIT license and update README with license link                         │
│                                                                                             │
│   - Added a new LICENSE file containing the full MIT License text                           │
│   - Updated README.md to replace placeholder license text with a link to the LICENSE file"  │
│ Score: 9/10                                                                                 │
│ Why it's good: Proper type 'chore' used; clear and descriptive message with useful details. │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────╮
│ Commit: "docs: enhance README with detailed usage, features, examples, and architecture" │
│ Score: 8/10                                                                              │
│ Why it's good: Uses 'docs' type correctly and clearly describes the changes made.        │
╰──────────────────────────────────────────────────────────────────────────────────────────╯

━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 YOUR STATS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━

┌──────────────────┬───────────┐
│ Average score    │ 6.8/10    │
│ Vague commits    │ 1 (25.0%) │
│ One-word commits │ 0 (0.0%)  │
└──────────────────┴───────────┘

Interactive mode

Analyzing staged changes... (2 files changed, +22 -1 lines)
╭─────────────────────────────────────────────────────────────────────────────────────────╮
│ SUMMARY:                                                                                │
│ - Add MIT License file with full text                                                   │
│ - Update README to link to the new LICENSE file                                         │
│                                                                                         │
│ SUBJECT:                                                                                │
│ chore: add MIT license and update README with license link                              │
│                                                                                         │
│ BODY:                                                                                   │
│ - Added a new LICENSE file containing the full MIT License text                         │
│ - Updated README.md to replace placeholder license text with a link to the LICENSE file │
╰─────────────────────────────────────────────────────────────────────────────────────────╯

Changes detected:
- Add MIT License file with full text
- Update README to link to the new LICENSE file

Suggested commit message:
╭─────────────────────────────────────────────────────────────────────────────────────────╮
│ chore: add MIT license and update README with license link                              │
│                                                                                         │
│ - Added a new LICENSE file containing the full MIT License text                         │
│ - Updated README.md to replace placeholder license text with a link to the LICENSE file │
╰─────────────────────────────────────────────────────────────────────────────────────────╯
Press Enter to accept, or type your own message ():

---

📐 Scoring rubric

CommitLens scores each commit from 0 to 10 based on Conventional Commit clarity and specificity.

Score bands:

• 0-4 -> needs_work
• 5-7 -> mid
• 8-10 -> well_written

What the stats mean:

• Average score: arithmetic mean of all analyzed commit scores
• Vague commits: commits with score < 5 (same threshold as needs_work)
• One-word commits: commits whose message contains only one word

---

🧪 Evaluation

Run eval suite:

uv run python evals/run_eval.py

Eval report includes:

• Bucket accuracy (needs_work / mid / well_written)
• Score tolerance metric (±1) for LLM variance
• Repeatability check: each eval case is run 5 times to measure consistency across runs

Eval bucket definitions:

• needs_work: score < 5
• mid: score 5-7
• well_written: score >= 8

How to read eval columns:

• Expected: expected bucket from evals/commits.json
• Expected Score: target score from evals/commits.json
• Pass %: percentage of runs where predicted bucket matched expected bucket
• Score μ: mean predicted score across repeated runs
• Score σ: score standard deviation across repeated runs
• Tolerance %: percentage of runs where abs(predicted - expected_score) <= 1

---

✅ Tests

Run tests:

python -m unittest discover -s tests -v

---

🧱 Architecture

git commits/diff
    -> prompt construction
    -> OpenAI LLM call
    -> Pydantic validation
    -> scoring + stats
    -> Rich terminal rendering

Key modules:

• commit_critic/app.py: CLI entry and mode orchestration
• commit_critic/git_ops.py: git cloning/log/diff utilities
• commit_critic/llm_client.py: LLM prompts, API calls, parsing, validation
• commit_critic/scoring.py: thresholds and statistics
• commit_critic/ui.py: rich output rendering
• evals/run_eval.py: lightweight scoring evaluation harness

---

⚙️ Options

• --limit: number of commits to analyze (default: 50)
• --model: OpenAI model to use (default: gpt-4.1-mini). See available OpenAI models
• --url: analyze a remote repository
• --analyze: analyze commit history mode
• --write: interactive commit writer mode

---

🌐 Remote repo behavior

• Uses shallow clone: git clone --depth 200
• Clones into a temporary directory for analysis
• Temporary clone directory is deleted after the run
• Best used with --limit to keep analysis focused on recent commits
• Public repositories are supported by default; private repositories require pre-configured Git credentials

---

🎯 Current scope

• OpenAI models are supported
• Remote --url analysis supports public Git repositories

---

🔮 Future work

• Multi-provider support (Anthropic/Gemini)
• Local model support
• Optional caching for repeated analyses

---

📂 Project structure

.
├── commit_critic/          # Core package logic
│   ├── app.py              # Typer CLI mode orchestration (--analyze / --write)
│   ├── config.py           # .env loading and API key validation
│   ├── git_ops.py          # Git operations (clone/log/diff/repo checks)
│   ├── llm_client.py       # LLM prompts, API calls, response parsing
│   ├── models.py           # Pydantic schemas for critiques/suggestions
│   ├── scoring.py          # Commit bucket logic and aggregate stats
│   ├── ui.py               # Rich terminal rendering
│   └── diff_cleaner.py     # Diff filtering/truncation for prompt safety
├── tests/                  # Unit tests (logic and mocked integrations)
├── evals/                  # LLM scoring evaluation harness
├── commit_critic.py        # CLI entry point
├── pyproject.toml          # Project metadata and dependencies
├── uv.lock                 # Reproducible dependency lockfile
├── .env.example            # Environment variable template
├── README.md               # Documentation
└── LICENSE                 # MIT license

---

License

MIT License