Run codey-video from OpenCode

[mike-holcomb]

User Requirement

The user wants to run the codey-video plugin (three video‑analysis skills) from the OpenCode platform instead of Claude Code. This requires exposing frame extraction, audio transcription, and multimodal answering via OpenCode’s plugin system.

Research Summary

• API Differences: Claude Code uses SKILL.md prompts that fork a Haiku sub‑agent and rely on Bash & Read tools. OpenCode plugins are TypeScript/JavaScript modules exporting a plugin function and interacting with the SDK (client.models.chat, client.files.upload, $ shell helper).
• Requirements: Access to the same preprocessing scripts (scripts/*.ts), Whisper models in resources/models, and a vision‑capable LLM (Claude 3.5 Sonnet, GPT‑4V, Gemini‑1.5‑Flash, etc.).
• Acceptance Criteria:
  1. The three skills (query-video-segment, generate-video-outline, generate-transcript) are callable from OpenCode (opencode run …).
  2. Results are comparable to the Claude Code version (frame count, transcript accuracy, multimodal answer quality).
  3. All processing stays local (ffmpeg, Whisper); no external services required.
  4. Installation works via opencode plugin install or by copying the plugin into .opencode/plugins.
• Testing Plan:
  • Unit Tests: Mock the SDK to verify correct chat message construction and file uploads.
  • Integration Tests: Run each skill on a small sample video (≤ 10 s) and assert:
    • At least one frame image is uploaded.
    • A non‑empty transcript JSON is produced.
    • The LLM response contains the user query text.
  • CI: GitHub Actions that install ffmpeg, execute opencode run for each skill, and compare output length to a baseline.

Additional Notes

• Expose only the needed bash and read tools (or a custom wrapper) to retain the security model.
• Model selection should be configurable via an environment variable (VISION_MODEL).
• Optional OpenCode hooks (notifications, logging) can be added later.

[mike-holcomb]

Migration Plan (high level)

• Assumptions: OpenCode provides a vision‑capable model; ffmpeg and Whisper can run locally; the repository’s symlink layout works on the target OS (or will be replaced with real copies for Windows).
• Design Decisions:
  • Implement a single OpenCode plugin that exports three tools (codey-video:query-video-segment, codey-video:generate-video-outline, codey-video:generate-transcript).
  • Re‑use the existing scripts/*.ts unchanged; invoke them via the $ shell helper.
  • Upload generated frame WebP files and transcript JSON via client.files.upload and include them in a multimodal chat request.
  • Use an environment variable VISION_MODEL to let the user pick any supported vision model (Claude, OpenAI, Google, etc.).
  • Keep a minimal custom Bash wrapper that only allows the three whitelist scripts to maintain the security posture of the original plugin.
• Key Limitations:
  • Image‑count per request is limited by the chosen vision model (usually ≤ 10‑20 images). Larger videos will need chunked requests.
  • Windows users may need to replace the symlinks with actual copies or use the file: npm linking approach.
  • Performance depends on local CPU for Whisper transcription; no GPU acceleration is assumed.
  • The migration does not yet add OpenCode‑specific hooks (notifications, logging) – they can be added in a later iteration.

Acceptance Criteria (re‑stated)

1. All three skills are invocable from OpenCode and produce output comparable to the Claude version.
2. No external services beyond the chosen vision LLM are required.
3. Installation works via opencode plugin install or by placing the plugin in .opencode/plugins.
4. CI runs the integration tests successfully on each push.

[mike-holcomb]

Repository‑organization considerations & repo‑splitting decision

How the migration reshapes the current layout

codey-video/
├─ .opencode/
│   ├─ plugins/
│   │   └─ codeyVideo.ts          ← OpenCode plugin entry point
│   └─ opencode.json              ← plugin manifest
├─ scripts/                       ← unchanged preprocessing utilities
├─ resources/models/               ← Whisper binaries (unchanged)
├─ skills/                        ← optional – keep for backward‑compatibility or archive
├─ README.md
├─ CLAUDE.md
└─ ...

• The three skills/*/SKILL.md files become obsolete; they can be removed or kept as documentation only.
• scripts/ and resources/ stay exactly where they are – the OpenCode plugin just points to them.
• A small addition (.opencode/) introduces the OpenCode manifest (opencode.json) and the TypeScript plugin file.

When to keep everything in this repo

• The plugin only re‑uses the existing scripts – a single git pull updates both Claude‑Code and OpenCode flows.
• The user base is still primarily Claude‑Code; OpenCode support is an added feature, not a separate product line.
• Simpler CI: one pipeline builds the scripts, runs the OpenCode integration tests, and publishes a single release that works for both ecosystems.
• No need to duplicate the large Whisper model binaries.

When a separate repository makes sense

• You want independent versioning (e.g., publishing the OpenCode plugin on npm as @my-org/codey-video-plugin) while the core repo continues its Claude‑Code‑only releases.
• The OpenCode community will treat this as a standalone plugin – a dedicated repo improves discoverability and encourages external contributions.
• Different licensing or distribution policies between the Claude‑Code manifest and the OpenCode plugin.
• Separate CI pipelines: a lightweight public CI for the OpenCode plugin vs. a heavier internal CI for Claude‑Code.

Practical recommendation for this project

1. Add the .opencode/ folder now and update the top‑level README with an OpenCode usage section.
2. Add a GitHub Actions workflow (opencode.yml) that installs ffmpeg, runs each skill via opencode run … on a tiny test video, and fails on empty output.
3. Tag the next release (e.g., v1.3‑opencode) indicating dual‑platform compatibility.
4. Monitor community traction. If OpenCode users start filing feature requests, PRs, or you need separate release cadence, spin the plugin off into its own repo (a thin wrapper that depends on the original repo’s scripts/ via a file: or submodule reference).
5. If you later split, keep the heavy preprocessing code in the original repository and let the new plugin repo import it, avoiding duplication of the Whisper model assets.

Bottom line – start with an in‑repo addition. It’s low‑risk, keeps the codebase coherent, and lets you evaluate real OpenCode adoption before committing to a separate repository.