feat(12): folder management, file type presets, and include_types pipeline by RichardHightower · Pull Request #113 · SpillwaveSolutions/agent-brain

RichardHightower · 2026-02-26T06:29:52Z

Summary

Folder management — CLI (folders list/add/remove) and API (GET/DELETE /index/folders) for tracking and managing indexed folders with JSONL persistence across server restarts
File type presets — Named preset system (python, docs, code, web, etc.) that resolves to glob patterns, with CLI types list command and --include-type flag on index
include_types pipeline fix — Threaded include_types through the full JobRecord → job_worker → DocumentLoader pipeline where data was silently dropped at 3 breakpoints. Added early preset validation (HTTP 400 for unknown presets)
Storage protocol extensions — Added delete_by_metadata() to StorageBackendProtocol for folder removal across Chroma and Postgres backends
Gitignore cleanup — Added rules for runtime state (.claude/agent-brain/), temp dirs (tmp/, .tmp-*/), and backup files (*.bak)

Implementation (3 waves + gap closure)

Wave	Focus	Key files
12-01	Server foundation — FolderManager, models, presets, storage protocol	`services/folder_manager.py`, `models/folders.py`, `services/file_type_presets.py`, `storage/protocol.py`
12-02	API integration — folders router, indexing service wiring	`api/routers/folders.py`, `services/indexing_service.py`, `api/main.py`
12-03	CLI + plugin — folders/types commands, `--include-type` flag	`commands/folders.py`, `commands/types.py`, `client/api_client.py`
Gap closure	Pipeline plumbing — JobRecord field, worker pass-through, DocumentLoader filtering, early validation	`models/job.py`, `job_queue/job_service.py`, `job_queue/job_worker.py`, `indexing/document_loader.py`, `api/routers/index.py`

Test plan

task before-push passes (format, lint, typecheck, all tests)
Server tests: 29 include_types tests, folder manager tests, folders API tests, file type preset tests
CLI tests: 121 tests pass including folders CLI, types CLI, fixed test_config isolation
UAT: 11/11 tests pass (4 gaps identified and resolved)

🤖 Generated with Claude Code

…ntent Pipeline 41 requirements across 4 categories (Folder Management, File Type Presets, Chunk Eviction, Content Injection) mapped to Phases 12-14. Research complete with architecture patterns, pitfall analysis, and feature landscape. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…t and file type presets

- FolderRecord dataclass with folder_path, chunk_count, last_indexed, chunk_ids - FolderManager with asyncio.Lock + atomic temp-file-rename for safe writes - Path normalization via Path.resolve() on all inputs - initialize() loads existing JSONL on startup gracefully - add_folder(), remove_folder(), list_folders(), get_folder(), clear() operations - 13 tests covering init, add, remove, list, persistence, concurrency, corrupt lines - Fixed test_get_folder_normalizes_path to use absolute path (relative CWD mismatch)

- FILE_TYPE_PRESETS: 14 presets (python, javascript, typescript, go, rust, java, csharp, c, cpp, web, docs, code, text, pdf) - resolve_file_types(): deduplicates patterns, raises ValueError with valid presets on unknown name - list_presets(): returns copy of preset dict for display - FolderInfo, FolderListResponse, FolderDeleteRequest, FolderDeleteResponse models - Export new symbols from services/__init__.py and models/__init__.py - 26 tests covering all 14 presets, deduplication, error handling, list_presets

…metadata - StorageBackendProtocol.delete_by_metadata(where) method added to protocol - VectorStoreManager.delete_by_where(): two-step query+delete with empty-ID guard - ChromaBackend.delete_by_metadata(): delegates to VectorStoreManager.delete_by_where - PostgresBackend.delete_by_metadata(): DELETE...RETURNING with JSONB containment filter - Fixed MockCompleteBackend in test_protocol.py to implement delete_by_metadata - Auto-fixed ruff linting issues in folder_manager.py (unused Any import, open mode) - All 725 tests pass, mypy strict clean, ruff+black clean

- 12-01-SUMMARY.md: FolderManager, FileTypePresets, delete_by_metadata summary - STATE.md: advance to plan 02, add phase 12 decisions, record 35min execution

- Create GET /index/folders and DELETE /index/folders endpoints - Add FolderManager initialization in server lifespan (project + temp dir modes) - Wire FolderManager into IndexingService for post-indexing folder registration - Add include_types field to IndexRequest with preset resolution - Add delete_by_ids to StorageBackendProtocol (ChromaDB + PostgreSQL backends) - Add VectorStoreManager.delete_by_ids with empty-ID guard - Update IndexingService.reset() to clear FolderManager records - Update IndexingService.get_status() to use FolderManager for persistent folder list - Pass include_types through index.py router resolved_request construction - Add 31 new tests for folders API and include_types functionality

- Folders API router with GET/DELETE /index/folders - FolderManager wired into server lifespan and IndexingService - include_types field on IndexRequest with preset resolution - delete_by_ids protocol extension for targeted folder chunk removal - 756 tests passing, mypy strict clean Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add FolderInfo dataclass to DocServeClient - Add list_folders() and delete_folder() methods to DocServeClient - Add include_types param to DocServeClient.index() - Create folders command group (list, add, remove) with Rich table output - Create types command group (list) with hardcoded presets (local, no server) - Register folders_group and types_group in cli.py - Add 35 tests for folders and types CLI commands (all pass) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…mands - Add --include-type flag to index command for file type presets - Parse comma-separated presets and pass include_types to server - Show include types in Rich output when specified - Create agent-brain-folders.md plugin slash command - Create agent-brain-types.md plugin slash command Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Remove unused imports (pytest, FolderRecord, FolderInfo, Path) - Remove unused local variable assignment in test_folders_api.py - Fix unnecessary open mode parameter in test_folder_manager.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… slash commands Phase 12 Plan 03 complete. All 3 tasks done: - Task 1: DocServeClient extensions, folders/types CLI commands, 35 tests - Task 2: --include-type flag for index, plugin slash commands - Task 3: Full quality gate passes (756 server + 121 CLI tests) Phase 12 (Folder Management & File Type Presets) COMPLETE. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Thread include_types through the full JobRecord → job_worker → DocumentLoader pipeline where data was silently dropped at 3 breakpoints. Add early preset validation in API endpoints (HTTP 400 for unknown presets). Fix CLI test isolation for get_server_url() that leaked stale runtime.json state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…aining phase 12 files Ignore .claude/agent-brain/ (server runtime state), tmp/, .tmp-*/ (manual test data), and *.bak files. Include remaining phase 12 docs, e2e scenario updates, and test file changes that were sitting unstaged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…ate todo - Move fix-include-types-pipeline-plumbing to done/ (resolved via PR #113) - Add new todo: update agent plugin and skills for Phase 12 features - Update STATE.md session continuity and pending todos Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

RichardHightower and others added 17 commits February 23, 2026 15:00

docs: start milestone v7.0 Index Management & Content Pipeline

7202d25

docs(12): research folder management & file type presets

0822a5c

docs(12): create phase plan — 3 plans in 3 waves for folder managemen…

2b772e6

…t and file type presets

docs(12-01): complete server foundation for folder management plan

2bce0ef

- 12-01-SUMMARY.md: FolderManager, FileTypePresets, delete_by_metadata summary - STATE.md: advance to plan 02, add phase 12 decisions, record 35min execution

docs: capture todo - Fix include_types pipeline plumbing

3678450

RichardHightower temporarily deployed to ci-testing February 26, 2026 06:29 — with GitHub Actions Inactive

RichardHightower merged commit b3a5be4 into main Feb 26, 2026
3 checks passed

RichardHightower deleted the feature/phase-12-folder-management-file-type-presets branch February 26, 2026 06:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(12): folder management, file type presets, and include_types pipeline#113

feat(12): folder management, file type presets, and include_types pipeline#113
RichardHightower merged 17 commits intomainfrom
feature/phase-12-folder-management-file-type-presets

RichardHightower commented Feb 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

RichardHightower commented Feb 26, 2026

Summary

Implementation (3 waves + gap closure)

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant