Skip to content

ksgisang/AI-Watch-Tester

BranchesTags

Repository files navigation

AWT Logo
AWT — AI Watch Tester
Enter a URL. AI generates, executes, and heals E2E tests — automatically.

CI MIT License Python 3.11+ Cloud GitHub Stars

Demo

AWT Demo
Enter a URL → AI scans your site → generates test scenarios → executes with live screenshots → reports results.

Why AWT?

Most E2E testing tools still require you to write code or record flows before you can run a single test. AWT flips that model:

  1. You provide a URL (and optionally a spec document).
  2. AI analyzes the page structure, forms, navigation, and auth flows.
  3. AI generates complete YAML test scenarios with selectors, test data, and assertions.
  4. Playwright executes the scenarios in a real browser with humanized input.
  5. If a test fails, the DevQA Loop kicks in — AI reads the failure, fixes the scenario or source code, and re-runs.

No test code. No recording. No manual maintenance.

Cloud Version vs Local CLI

AWT ships in two modes. Pick one — or use both.

Cloud Local CLI
URL ai-watch-tester.vercel.app aat dashboard → localhost:9500
Install None — just sign up pip install aat-devqa
Browser Headless Chromium on server Real Chromium on your machine
AI key Server-provided or BYOK Your own key (OpenAI / Anthropic / Ollama)
Best for PMs, planners, quick tests Developers, CI/CD, offline use
Pricing Free (5/mo) · Pro $28.99 · Team $98.99 Free forever (MIT, unlimited)
Data Stored on our servers Never leaves your machine

Cloud — Start in 30 seconds

1. Visit https://ai-watch-tester.vercel.app
2. Sign up (email or GitHub)
3. Enter your target URL
4. Watch AI generate and execute tests

Local CLI — Full control

pip install aat-devqa
playwright install chromium

# Option 1: Web dashboard
aat dashboard                    # http://localhost:9500

# Option 2: CLI
aat start                        # guided mode
aat generate --url https://example.com --provider openai
aat run scenarios/

From Source

git clone https://github.com/ksgisang/AI-Watch-Tester.git
cd AI-Watch-Tester
python -m venv .venv && source .venv/bin/activate
make dev          # install deps + playwright + pre-commit
make test         # verify everything works
aat dashboard     # launch web UI

Features

Feature Description
🤖 AI Scenario Generation Upload a URL or spec doc (PDF/DOCX/MD) — AI creates E2E test scenarios
🌐 Real Browser Testing Playwright-driven Chromium with Bezier mouse curves and variable-speed typing
♻️ Self-Healing DevQA Loop AI analyzes failures, patches code or scenarios, and re-runs automatically
☁️ Cloud + Local Cloud mode (no install, browser dashboard) or local mode (real browser, full control)
📊 Live Dashboard Real-time screenshot streaming, step-by-step progress, event log
📄 Document-Based Generation Feed PDF/DOCX/Markdown specs — AI generates scenarios from requirements
🔑 BYOK Bring your own AI API key (OpenAI, Anthropic, Ollama) — encrypted at rest
🧪 CI/CD Ready One-line curl integration with any pipeline
🧩 Plugin Architecture Engines, matchers, AI adapters, and reporters are all swappable via registries

Supported AI Providers

Provider Models Cost Setup
OpenAI gpt-4o, gpt-4o-mini Pay-per-use export OPENAI_API_KEY=sk-...
Anthropic Claude Sonnet 4 Pay-per-use export ANTHROPIC_API_KEY=sk-ant-...
Ollama codellama, llama3, mistral Free (local GPU) ollama serve

Configure in aat.yaml or via environment variables:

ai:
  provider: openai        # openai | anthropic | ollama
  model: gpt-4o
  api_key: ${OPENAI_API_KEY}

Cloud users can bring their own API key (BYOK) via Settings > AI Provider.

How It Compares

vs testRigor

AWT testRigor
Test authoring AI generates from URL/docs — zero input Plain English commands (you write)
Self-healing DevQA Loop (AI re-generates) Built-in auto-maintenance
Pricing Free (MIT, self-host) Enterprise pricing (~$800+/mo)
Open source Yes No
Setup time Seconds (enter URL) Minutes (write English scripts)

Choose AWT if you want fully automated test generation with no scripting at all, or need a self-hostable open-source tool. Choose testRigor if you prefer writing plain-English test specs with enterprise support.

vs Applitools

AWT Applitools
Primary focus Functional E2E test generation + execution Visual regression + cross-browser comparison
AI role Generates entire test scenarios Compares screenshots for visual differences
Standalone Yes (full pipeline) No (requires Cypress/Playwright/Selenium)
Pricing Free (MIT) Free tier + paid plans

Choose AWT for AI-driven functional testing where you need scenarios generated automatically. Choose Applitools when pixel-perfect visual consistency across browsers is the priority. They complement each other — AWT generates and runs tests, Applitools can validate visual output.

vs Playwright / Cypress

These are excellent browser automation frameworks that AWT is built on top of. The difference is who writes the tests: you (Playwright/Cypress) or AI (AWT). If your team wants full programmatic control, use them directly. If you want AI to handle test creation and maintenance, AWT fills that gap.

See docs/COMPARISON.md for a detailed breakdown against Playwright, Cypress, Testim, Katalon, and Mabl.

Architecture

aat start / aat dashboard
       │
       ▼
┌─────────────────────────────────────────────┐
│                   CLI (Typer)                │
├─────────────────────────────────────────────┤
│              Core Orchestrator              │
│  ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│  │ Executor │ │Comparator│ │ DevQA Loop  │ │
│  └────┬─────┘ └────┬─────┘ └──────┬──────┘ │
├───────┼─────────────┼──────────────┼────────┤
│  ┌────▼────┐  ┌─────▼─────┐  ┌────▼─────┐  │
│  │ Engine  │  │  Matcher  │  │ Adapter  │  │
│  │Registry │  │ Registry  │  │ Registry │  │
│  └─────────┘  └───────────┘  └──────────┘  │
│  web|desktop  template|ocr   openai|claude  │
│               feature|hybrid ollama         │
├─────────────────────────────────────────────┤
│  Models (Pydantic v2)  │  Config (Settings) │
└─────────────────────────────────────────────┘

All modules follow ABC + plugin registry pattern — extend the base class, register in __init__.py, done.

Development

Prerequisites

  • Python 3.11+
  • Tesseract OCR — brew install tesseract / apt install tesseract-ocr
  • Git

Make Commands

Command Description
make dev Install all deps + Playwright + pre-commit
make lint ruff check
make format ruff format + auto-fix
make typecheck mypy strict
make test pytest
make test-cov pytest + coverage report
make clean Remove caches and build artifacts

Contributing

Contributions are welcome! See CONTRIBUTING.md for:

  • Development environment setup
  • Code style (ruff + mypy strict)
  • Test writing guidelines
  • Pull request process
  • Plugin development (adding new engines, matchers, or AI adapters)
git checkout -b feat/my-feature
# make changes
make format && make lint && make typecheck && make test
git commit -m "feat(scope): description"
# open PR

Documentation

Document Description
Quick Start Guide Install, configure, run your first test
API Reference REST API + WebSocket documentation
Comparison AWT vs Playwright, Cypress, Testim, Katalon, Mabl
FAQ Common questions
CI/CD Guide Pipeline integration (GitHub Actions, GitLab CI)
Cloud Backend Self-hosting the cloud backend

FAQ

What is AWT?
AWT (AI Watch Tester) is an open-source, AI-powered E2E testing tool. You give it a URL, and it automatically generates test scenarios, executes them in a real browser (Playwright), and reports results — no test code required.
How do I install it?

Cloud (no install): Visit ai-watch-tester.vercel.app and enter a URL.

Local:

pip install aat-devqa
playwright install chromium
aat dashboard

From source:

git clone https://github.com/ksgisang/AI-Watch-Tester.git
cd AI-Watch-Tester
make dev && aat dashboard
Which AI providers are supported?
Provider Models Cost
OpenAI gpt-4o, gpt-4o-mini Pay-per-use
Anthropic Claude Sonnet 4 Pay-per-use
Ollama codellama, llama3, mistral Free (local GPU)

Cloud users can bring their own API key (BYOK) via the Settings page. Keys are Fernet-encrypted at rest.

How much does it cost?
Plan Price Tests/month Concurrent
Free $0 5 1
Pro $28.99/mo 100 3
Team $98.99/mo 500 10

The open-source local mode is completely free with no limits — you just need your own AI API key.

Is it open source?
Yes. AWT is licensed under the MIT License — free for personal and commercial use. You can self-host, modify, and distribute it. Contributions are welcome!
Can I use it in CI/CD?

Yes. Pro and Team plans include API keys for CI/CD integration:

curl -X POST https://your-awt-server.com/api/v1/run \
  -H "X-API-Key: awt_your_key" \
  -H "Content-Type: application/json" \
  -d '{"target_url": "https://staging.example.com"}'

See the CI/CD Guide for GitHub Actions and GitLab CI examples.

Is my data secure?
  • All traffic is encrypted via HTTPS/TLS
  • BYOK API keys are Fernet-encrypted (AES-128-CBC + HMAC-SHA256) at rest
  • Screenshots are auto-deleted after 7 days
  • Database hosted on Supabase (AWS Seoul region)
  • See our Privacy Policy for full details

License

MIT — free for personal and commercial use.

Built with Playwright, OpenCV, and a lot of AI. Made by @ksgisang.

About

AI-powered automated E2E testing. Just enter a URL — AI generates and runs test scenarios.

ai-watch-tester.vercel.app

Topics

open-source test-automation e2e-testing playwright ai-qa ai-testing no-code-testing

Resources

Readme

License

MIT license

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 1

🚀 AWT v1.0.0 — AI-Powered E2E Test Generator Latest
Feb 24, 2026

Packages

 
 
 

Contributors

Languages