Skip to content

bug-first test prompt #6

@bitkojine

Description

@bitkojine

You are a coding agent improving a test suite using a bug-first approach.

Goal:
Ensure all tests meaningfully protect against real bugs and prove their effectiveness via mutation.

Rules:

  • Every test must begin by answering: “What bug does this cover?”
  • If you cannot name a specific, realistic bug, do not write the test.
  • Each test must cover exactly one bug.
  • Avoid happy-path-only or “no crash” tests.
  • Prefer edge cases, boundaries, invalid inputs, state transitions, and regressions.

Procedure:

  1. Explore the repo and existing tests.
  2. Find one test that “tests nothing” (would still pass if the code were obviously wrong).
    • If found, remove and replace exactly one such test.
    • If none found, add exactly one new high-quality bug-first test.
  3. For the new test:
    • State the bug it covers (as a comment above the test).
    • Briefly explain how the bug could realistically occur.
    • Write the minimal test that would fail if the bug exists.
  4. Mutation verification (mandatory):
    • Intentionally mutate production code to introduce the described bug.
    • Run tests and confirm the new test fails for the intended reason.
    • Restore/fix the code and confirm all tests pass again.
  5. Leave the repo in a fully passing state.

Completion criteria:

  • Exactly one test replaced or added.
  • The test clearly names a real bug.
  • The test fails under mutation and passes when fixed.
  • Both failing and passing states were observed.

Principle:
If you cannot explain the failure a test prevents, the test must not exist.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions