Skip to content

Audit your AI agent setup for performance, cost, and ROI. Works with any model provider.

Notifications You must be signed in to change notification settings

bokiko/agent-audit

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Agent Audit

Cut your AI agent costs by 40-60% without losing quality.

Most people running AI agents are overpaying. Opus on a health check. GPT-4 on a reminder. That's $75/M tokens on a task a $0.25 model handles perfectly.

Agent Audit scans your entire OpenClaw setup and tells you exactly where you're bleeding money — and what to swap.

What It Does

  • Scans every cron job, agent, and model assignment in your config
  • Classifies tasks by complexity — simple tasks get cheap models, complex ones stay protected
  • Recommends specific swaps with estimated monthly savings and risk levels
  • Works with any provider — Claude, GPT, Gemini, Grok
  • Never downgrades coding, security, or trading tasks

Install

npx clawhub install agent-audit

Usage

Once installed, just tell your agent:

"audit my agents" "where am I wasting tokens" "optimize my AI costs"

Or run the script directly:

python3 scripts/audit.py                    # Full markdown report
python3 scripts/audit.py --format summary   # Quick savings number
python3 scripts/audit.py --output report.md # Save to file

Example Output

💰 SAVE $12.40/mo: Swap "Health Check" from opus → haiku
   Current: $15/M input, $75/M output
   Suggested: $0.25/M input, $1.25/M output
   Risk: LOW | Confidence: HIGH

💰 SAVE $8.20/mo: Swap "Weather Check" from sonnet → haiku
   Current: $3/M input, $15/M output
   Suggested: $0.25/M input, $1.25/M output
   Risk: LOW | Confidence: HIGH

Supported Providers

Provider Models
Anthropic Opus, Sonnet, Haiku
OpenAI GPT-4.5, GPT-4o, GPT-4o-mini, o1, o3-mini
Google Gemini 2.5 Pro, 2.0 Flash, Flash-Lite
xAI Grok-3, Grok-3-mini, Grok-4

Safety Rails

The audit never recommends downgrading:

  • Coding and development tasks
  • Security reviews and audits
  • Trading and financial tasks
  • Tasks that previously failed on weaker models
  • Your main interactive session model

How It Works

  1. Discovery — reads your OpenClaw config, maps every agent and cron job to its model and pricing
  2. Analysis — pulls run history, calculates per-job token usage, runtime, and success rate
  3. Classification — categorizes each task as simple, medium, or complex based on name patterns, output length, and runtime
  4. Recommendations — flags every mismatch where you're paying more than you need to, with confidence scores and risk levels

Requirements

  • OpenClaw installed and configured
  • Python 3.8+

License

MIT — free and open source.

About

Audit your AI agent setup for performance, cost, and ROI. Works with any model provider.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 100.0%