diff --git a/public/blog/forge-blog-banner.png b/public/blog/forge-blog-banner.png new file mode 100644 index 0000000..d81c156 Binary files /dev/null and b/public/blog/forge-blog-banner.png differ diff --git a/src/content/blog/forge-built-for-enterprise-work.md b/src/content/blog/forge-built-for-enterprise-work.md new file mode 100644 index 0000000..562dd03 --- /dev/null +++ b/src/content/blog/forge-built-for-enterprise-work.md @@ -0,0 +1,264 @@ +--- +title: "OpenClaw Demonstrated Agents Can Act. The Next Question Is: Can They Operate in an Enterprise?" +description: "AI agents are no longer hypothetical. But personal agents and enterprise agents are fundamentally different. Forge is the secure, scalable, enterprise-grade runtime for AI work." +author: "Forge Team" +date: 2026-03-04 +tags: ["enterprise", "security", "launch", "openclaw"] +image: "/blog/forge-blog-banner.png" +--- + +OpenClaw made one thing clear: + +**AI agents are no longer hypothetical.** + +They can: + +- Navigate environments +- Plan multi-step actions +- Interact with real systems +- Execute workflows autonomously + +That's a real milestone. + +But it exposes a harder question. + +**What changes when this leap moves from personal assistants to enterprise operations?** + +Imagine an agent: + +- Closing quarterly financial books +- Extracting sensitive ERP data +- Reconciling accounts +- Generating compliance-ready reports +- Routing approvals across departments + +Now the stakes are different. + +This is no longer a productivity tool. + +**This is operational infrastructure.** + +## Personal Agents vs Enterprise Agents + +OpenClaw demonstrates personal agents extremely well. + +Personal agents optimize for: + +- Speed +- Autonomy +- Exploration +- Capability +- Convenience + +They operate in user-controlled environments. + +Enterprise agents operate under different constraints. + +Enterprises optimize for: + +- Identity enforcement +- Tenant isolation +- Audit logging +- Controlled network egress +- Deployment portability +- Regulatory compliance + +**Moving from "assistant" to "enterprise worker" fundamentally changes the requirements.** + +An enterprise agent is not a browser macro. + +It behaves like a workload. + +## The Hidden Infrastructure Problem + +Most agent frameworks today are designed for demos. + +They often: + +- Expose inbound tunnels (ngrok-style) +- Open local ports for callbacks +- Execute arbitrary code without strong isolation +- Blur boundaries between local, staging, and production +- Lack structured identity and tenancy models + +That's acceptable for experimentation. + +It's untenable in a regulated organization. + +The next wave of agent failures will not be hallucinations. + +They will be: + +- Privilege escalation +- Cross-tenant data leakage +- Environment confusion (dev vs prod) +- Unbounded execution +- Uncontrolled outbound calls + +When an agent can restart infrastructure, rotate secrets, access databases, and call external APIs, it effectively becomes a privileged service account. + +And privileged service accounts require containment. + +This is not a prompt problem. + +**It's a runtime problem.** + +## Introducing Forge + +Forge is built for this shift. + +It is a secure, scalable, enterprise-grade runtime for AI agents. + +Not a demo layer. + +Not a wrapper. + +**Infrastructure.** + +Forge allows you to: + +- Run agents locally +- Deploy the same agent to a corporate VPC +- Operate in private cloud environments +- Run inside air-gapped clusters +- Avoid inbound tunnels entirely +- Maintain outbound-only communication models +- Preserve strict environment boundaries + +Without rewriting agent code. + +The same agent definition runs across environments — predictably. + +## The Principles Behind Forge + +### Atomicity + +Every execution is isolated and scoped. + +Skills define what an agent can do. Execution is bounded. Side effects are controlled. + +Agents behave like contained workloads — not autonomous chaos. + +Strong execution boundaries reduce blast radius. + +### Security + +Forge is outbound-first. + +No exposed ports. +No developer machine callbacks. +No hidden tunnels. + +Enterprise security teams require containment. + +Forge enforces it. + +### Portability + +The same agent runs: + +- On your laptop +- Inside your corporate VPC +- In private cloud +- In air-gapped enterprise clusters + +No re-architecture. + +No environment drift. + +Portability is not optional. It is foundational. + +### Governance + +Enterprise agents require: + +- Skill trust models +- Execution scoping +- Secret management +- Production policy enforcement + +Forge aligns agent execution with compliance expectations from day one. + +## Why This Moment Matters + +OpenClaw showed that agents can act. + +That's important. + +But as capability grows, the real question shifts from: + +*"Can it do this?"* + +to: + +*"Can it do this safely, predictably, and at organizational scale?"* + +A single hour of downtime in a large enterprise can cost hundreds of thousands of dollars. + +A compliance violation can cost millions. + +Unsafe automation at scale becomes a board-level issue. + +The agent revolution will not be limited by intelligence. + +**It will be limited by infrastructure.** + +## Personal AI Is Just the Beginning + +Personal AI will continue to accelerate innovation. + +But enterprises require more than capability. + +They require: + +- Boundaries +- Observability +- Identity +- Policy alignment +- Deployment control +- Portability guarantees + +Agents must graduate from demos to disciplined workloads. + +That transition demands a hardened runtime layer. + +## The Next Phase of Agents + +We believe the future is an enterprise AI workforce. + +Agents that: + +- Triage incidents +- Manage infrastructure +- Execute business workflows +- Integrate with Slack, Jira, GitHub +- Operate across environments +- Remain auditable and portable + +That future cannot rely on ad-hoc execution models. + +It requires runtime architecture. + +## Forge Is Open Source + +If you're building agents for production environments, Forge provides the infrastructure layer to run them securely. + +Clone the repository. +Run an agent locally. +Deploy it into your own environment. + +No inbound tunnels. +No environment drift. +No architectural compromises. + +OpenClaw showed what agents can do. + +**Forge ensures agents can operate when the work actually matters.** + +The agent revolution will not be won at the prompt layer. + +It will be won at the runtime layer. + +That's where Forge lives. + +[GitHub](https://github.com/initializ/forge) | [Website](https://useforge.ai) diff --git a/src/content/docs/core-concepts/channels.md b/src/content/docs/core-concepts/channels.md index 8a9f1b8..c181819 100644 --- a/src/content/docs/core-concepts/channels.md +++ b/src/content/docs/core-concepts/channels.md @@ -1,6 +1,6 @@ --- title: Channels -description: Connect your agent to Slack and Telegram — webhooks, polling, large response handling, and channel setup. +description: "Connect your agent to Slack and Telegram — Socket Mode, polling, mention-aware filtering, and large response handling." order: 4 editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/core-concepts/channels.md --- @@ -9,41 +9,56 @@ editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/cor Channels connect your agent to messaging platforms. Instead of interacting through the CLI or HTTP API, your users talk to the agent in Slack or Telegram, and the agent responds in the same thread. -Channels run **with** the agent process — they are not separate services. You start them with `forge serve --with `. +Channels run **with** the agent process — they are not separate services. You start them with `forge run --with ` or `forge channel serve `. ## Channel Overview | Channel | Mode | Default Port | Features | |---|---|---|---| -| Slack | Webhook | 3000 | HMAC-SHA256 signature verification, replay protection (5-min window), URL verification challenge, file upload for large responses | +| Slack | Socket Mode | 3000 | Outbound-only WebSocket, mention-aware filtering, processing indicators, file upload for large responses | | Telegram | Polling (default), Webhook | 3001 | Long polling (30s timeout), typing indicator, markdown-to-HTML conversion, document upload for large responses | -Both channels are designed for outbound-first operation — your agent processes incoming messages and sends responses back through the platform's API. +Both channels are designed for outbound-first operation — no public URLs or webhooks required. Your agent connects outbound to the platform's API. ## Slack -The Slack channel receives events via webhook on port 3000. When a user mentions your bot or sends it a direct message, Slack delivers the event to your agent's webhook endpoint. +The Slack channel uses **Socket Mode** — an outbound WebSocket connection to Slack's servers. This eliminates the need for public URLs, webhooks, or inbound tunnels. -### Security +### Setup + +```bash +forge channel add slack +``` -- **HMAC-SHA256 signature verification** — every incoming request is verified against your signing secret. Invalid signatures are rejected. -- **Replay protection** — requests older than 5 minutes are rejected to prevent replay attacks. -- **URL verification challenge** — Forge automatically responds to Slack's URL verification challenge during setup, so you do not need to handle it manually. +This runs an interactive setup that: +1. Creates a Slack App with Socket Mode enabled +2. Generates an app-level token with `connections:write` scope +3. Enables event subscriptions for `message.channels`, `message.im`, and `app_mention` +4. Adds bot token scopes: `app_mentions:read`, `chat:write`, `channels:history`, `im:history`, `files:write`, `reactions:write` +5. Validates tokens against the Slack API +6. Writes configuration to `forge.yaml` and secrets to `.env` ### Required Environment Variables | Variable | Purpose | |---|---| | `SLACK_BOT_TOKEN` | OAuth token for sending messages and uploading files | -| `SLACK_SIGNING_SECRET` | Used to verify incoming webhook signatures | +| `SLACK_APP_TOKEN` | App-level token for Socket Mode connection | -### Setup +### Mention-Aware Filtering -```bash -forge channel add slack -``` +The bot uses intelligent message filtering: + +- **Channel messages** — only responds to @mentions +- **Threads** — responds to all messages in threads it participates in +- **Direct messages** — responds to all messages + +### Processing Indicators -This runs an interactive setup that prompts you for your bot token and signing secret, validates them against the Slack API, and writes the configuration to `forge.yaml` and the secrets to `.env`. +For queries that take more than a few seconds, the bot provides visual feedback: + +- **Eyes reaction** — added to the message when processing starts +- **Status messages** — interim status updates posted for queries exceeding 15 seconds ## Telegram @@ -52,8 +67,8 @@ The Telegram channel uses long polling by default on port 3001. It periodically ### Features - **Long polling** — fetches updates with a 30-second timeout, keeping latency low without constant requests -- **Typing indicator** — sends a "typing..." indicator every 4 seconds while the agent is processing a response, so users know the agent is working -- **Markdown-to-HTML conversion** — the agent's markdown responses are converted to Telegram-compatible HTML for proper formatting +- **Typing indicator** — sends a "typing..." indicator every 4 seconds while the agent is processing a response +- **Markdown-to-HTML conversion** — the agent's markdown responses are converted to Telegram-compatible HTML ### Required Environment Variables @@ -74,11 +89,11 @@ This runs an interactive setup that prompts you for your bot token, validates it Start your agent with a channel connector: ```bash -forge serve --with slack +forge run --with slack ``` ```bash -forge serve --with telegram +forge run --with telegram ``` The channel starts **with** the agent as part of the same process. There is no separate channel service to deploy or manage. The agent listens on port 8080 for HTTP/SSE traffic and the channel listens on its own port (3000 for Slack, 3001 for Telegram). @@ -86,9 +101,20 @@ The channel starts **with** the agent as part of the same process. There is no s You can run multiple channels simultaneously: ```bash -forge serve --with slack --with telegram +forge run --with slack,telegram +``` + +### Standalone Mode + +Run a channel adapter separately from the agent: + +```bash +export AGENT_URL=http://localhost:8080 +forge channel serve slack ``` +This is useful when you want to run the agent and channel adapters as separate processes, for example in different containers. + ## Large Response Handling When an agent response exceeds 4096 characters, Forge uses a split-and-upload strategy instead of dumping a wall of text into the chat. @@ -102,15 +128,19 @@ When an agent response exceeds 4096 characters, Forge uses a split-and-upload st - **Telegram** — uses `sendDocument` with multipart upload 4. If the file upload fails, Forge falls back to **chunked messages** — splitting the response into chunks (4000 characters for Slack, 4096 for Telegram) and sending them sequentially +Large tool outputs are tracked and attached separately to preserve complete, untruncated data. + ### Telegram Retry Logic If `sendDocument` fails on Telegram, Forge retries without reply context (in case the original message was deleted or inaccessible). If that also fails, it falls back to chunked text messages. -This approach keeps chat threads clean — users get a concise summary inline and can open the full report as a document when they need the details. - ## Router Timeout -The channel router uses a 360-second timeout to accommodate long-running skills. Skills like deep research or incident triage can take several minutes, and the default HTTP timeout would cut them off. The 360-second window gives your agent enough time to complete complex tasks before the channel considers the request timed out. +The channel router uses a 360-second timeout to accommodate long-running skills. Skills like deep research or incident triage can take several minutes, and the default HTTP timeout would cut them off. + +## Custom Adapters + +Developers can create custom channel adapters by implementing the `ChannelPlugin` interface with methods for initialization, starting, stopping, event normalization, and response delivery. ## What's Next diff --git a/src/content/docs/core-concepts/hooks.md b/src/content/docs/core-concepts/hooks.md new file mode 100644 index 0000000..e6dbeda --- /dev/null +++ b/src/content/docs/core-concepts/hooks.md @@ -0,0 +1,121 @@ +--- +title: Hook System +description: "Extend the Forge agent loop with hooks — logging, enforcement, progress tracking, and audit events." +order: 7 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/core-concepts/hooks.md +--- + +# Hook System + +The hook system allows custom logic to run at key points in the LLM agent loop. Hooks can observe, modify context, or block execution. + +## Overview + +Hooks fire synchronously during the agent loop and can: + +- **Log** interactions for debugging or auditing +- **Block** execution by returning an error +- **Inspect** messages, responses, and tool activity + +## Hook Points + +| Hook Point | When It Fires | Available Data | +|-----------|---------------|------------------| +| `BeforeLLMCall` | Before each LLM API call | `Messages`, `TaskID`, `CorrelationID` | +| `AfterLLMCall` | After each LLM API call | `Messages`, `Response`, `TaskID`, `CorrelationID` | +| `BeforeToolExec` | Before each tool execution | `ToolName`, `ToolInput`, `TaskID`, `CorrelationID` | +| `AfterToolExec` | After each tool execution | `ToolName`, `ToolInput`, `ToolOutput`, `Error`, `TaskID`, `CorrelationID` | +| `OnError` | When an LLM call fails | `Error`, `TaskID`, `CorrelationID` | +| `OnProgress` | During tool execution | `Phase`, `ToolName`, `StatusMessage` | + +## HookContext + +The `HookContext` struct carries data available at each hook point: + +```go +type HookContext struct { + Messages []llm.ChatMessage // Current conversation messages + Response *llm.ChatResponse // LLM response (AfterLLMCall only) + ToolName string // Tool being executed + ToolInput string // Tool input arguments (JSON) + ToolOutput string // Tool result (AfterToolExec only) + Error error // Error that occurred +} +``` + +## Writing Hooks + +Hooks implement the `Hook` function signature: + +```go +type Hook func(ctx context.Context, hctx *HookContext) error +``` + +### Logging Hook Example + +```go +hooks := engine.NewHookRegistry() + +hooks.Register(engine.BeforeLLMCall, func(ctx context.Context, hctx *engine.HookContext) error { + log.Printf("LLM call with %d messages", len(hctx.Messages)) + return nil +}) + +hooks.Register(engine.AfterToolExec, func(ctx context.Context, hctx *engine.HookContext) error { + log.Printf("Tool %s returned: %s", hctx.ToolName, hctx.ToolOutput) + return nil +}) +``` + +### Enforcement Hook Example + +```go +hooks.Register(engine.BeforeToolExec, func(ctx context.Context, hctx *engine.HookContext) error { + if hctx.ToolName == "dangerous_tool" { + return fmt.Errorf("tool %q is blocked by policy", hctx.ToolName) + } + return nil +}) +``` + +## Audit Logging + +The runner registers `AfterLLMCall` hooks that emit structured audit events for each LLM interaction. Audit fields include: + +| Field | Description | +|-------|-------------| +| `provider` | LLM provider name | +| `model` | Model identifier | +| `input_tokens` | Prompt token count | +| `output_tokens` | Completion token count | +| `organization_id` | OpenAI Organization ID (when set) | + +These events are logged via `slog` at Info level and can be consumed by external log aggregators for cost tracking and compliance. + +## Progress Tracking + +The runner automatically registers progress hooks that emit real-time status updates during tool execution. Progress events include the tool name, phase (`tool_start` / `tool_end`), and a human-readable status message. These events are streamed to clients via SSE when using the A2A HTTP server, enabling live progress indicators in web and chat UIs. + +## Error Handling + +- Hooks fire **in registration order** for each hook point +- If a hook returns an **error**, execution stops immediately +- The error propagates up to the `Execute` caller +- For `BeforeToolExec`, returning an error prevents the tool from running +- For `OnError`, the error from the LLM call is available in `hctx.Error` + +## Registration + +```go +hooks := engine.NewHookRegistry() +hooks.Register(engine.BeforeLLMCall, myHook) +hooks.Register(engine.AfterToolExec, myOtherHook) + +exec := engine.NewLLMExecutor(engine.LLMExecutorConfig{ + Client: client, + Tools: tools, + Hooks: hooks, +}) +``` + +If no `HookRegistry` is provided, an empty one is created automatically. diff --git a/src/content/docs/core-concepts/how-forge-works.md b/src/content/docs/core-concepts/how-forge-works.md index 2849c18..42e7dd2 100644 --- a/src/content/docs/core-concepts/how-forge-works.md +++ b/src/content/docs/core-concepts/how-forge-works.md @@ -114,6 +114,28 @@ Forge is organized into focused modules with clear dependency boundaries: The dependency direction flows inward: `forge-cli` depends on `forge-core`, which depends on `forge-skills/contract`. No circular dependencies. The core library has no knowledge of CLI concerns, and channels are pluggable without modifying core. +### Key Go Interfaces + +| Interface | Purpose | +|---|---| +| `llm.Client` | Provider-agnostic LLM client — `Chat()`, `ChatStream()`, `ModelID()` | +| `runtime.AgentExecutor` | Contract for running agents — `LLMExecutor` is the primary implementation | +| `tools.Tool` | Tool definition — name, JSON Schema, and execution method | +| `runtime.ToolExecutor` | Tool execution interface for dependency injection | +| `channels.ChannelPlugin` | Messaging platform adapter — init, start, stop, event normalization | + +### Data Flow + +``` +Compilation: + ForgeConfig → Validation → Skill Compilation → AgentSpec + → Security Policy → Build Artifacts + +Runtime: + AgentSpec + Tools → LLM Client → Agent Loop + (prompt → LLM → tool calls → execution → results → response) +``` + ## Build Outputs Running `forge build` produces these artifacts in `.forge-output/`: diff --git a/src/content/docs/core-concepts/memory-system.md b/src/content/docs/core-concepts/memory-system.md index e960f78..a969035 100644 --- a/src/content/docs/core-concepts/memory-system.md +++ b/src/content/docs/core-concepts/memory-system.md @@ -23,7 +23,15 @@ The core `Memory` struct holds three things: ### Character Budget -Each model has a character budget derived from its context window. For example, `gpt-4o` with a 128K token context window gets a character budget of approximately 435K characters (applying an 85% safety margin to account for tokenization variance). +Each model has a character budget derived from its context window (applying an 85% safety margin for tokenization variance): + +| Model | Context Window | Character Budget | +|---|---|---| +| `gpt-4o` / `gpt-5` | 128K tokens | ~435K chars | +| `claude-sonnet` / `claude-opus` | 200K tokens | ~680K chars | +| `gemini-2.5` | 1M tokens | ~3.4M chars | +| `llama3` | 8K tokens | ~27K chars | +| `llama3.1` | 128K tokens | ~435K chars | The budget ensures your agent never exceeds the model's context window, even with long conversations and large tool outputs. @@ -186,11 +194,15 @@ Before the compactor discards old messages, the `MemoryFlusher` extracts key obs This ensures important context is not lost when session memory compacts. The flusher applies different extraction limits based on content type: -- **Research results** — up to 5000 characters extracted (research tends to contain dense, valuable information) +- **Research results** — up to 5000 characters extracted and tagged with `[research][tool:tavily_research]` so research insights persist distinctly across sessions - **General conversation** — up to 2000 characters extracted After flushing, the new observations are chunked and indexed for search. +### MEMORY.md Template + +When long-term memory is enabled, Forge creates a `.forge/memory/` directory with a `MEMORY.md` file for curated facts. This file serves as the agent's evergreen knowledge base — both the agent and you can edit it. Content in `MEMORY.md` is exempt from temporal decay in search, making it ideal for persistent facts, user preferences, and project context. + ## Graceful Degradation The memory system degrades gracefully when components are unavailable: diff --git a/src/content/docs/core-concepts/runtime-engine.md b/src/content/docs/core-concepts/runtime-engine.md new file mode 100644 index 0000000..3a99f40 --- /dev/null +++ b/src/content/docs/core-concepts/runtime-engine.md @@ -0,0 +1,210 @@ +--- +title: Runtime Engine +description: "How the Forge runtime engine powers agent execution — LLM providers, fallback chains, executors, and running modes." +order: 6 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/core-concepts/runtime-engine.md +--- + +# LLM Runtime Engine + +The runtime engine powers `forge run` — executing agent tasks via LLM providers with tool calling, conversation memory, and lifecycle hooks. + +## Agent Loop + +The core agent loop follows a simple pattern: + +1. **Initialize memory** with the system prompt and task history +2. **Append** the user message +3. **Call the LLM** with the conversation and available tool definitions +4. If the LLM returns **tool calls**: execute each tool, append results, go to step 3 +5. If the LLM returns a **text response**: return it as the final answer +6. If **max iterations** are exceeded: return an error + +``` +User message → Memory → LLM → tool_calls? → Execute tools → LLM → ... → text → Done +``` + +The loop terminates when `FinishReason == "stop"` or `len(ToolCalls) == 0`. + +## LLM Providers + +Forge supports multiple LLM providers with automatic fallback: + +| Provider | Default Model | Auth | +|----------|--------------|------| +| `openai` | `gpt-5.2-2025-12-11` | API key or OAuth; optional Organization ID | +| `anthropic` | `claude-sonnet-4-20250514` | API key | +| `gemini` | `gemini-2.5-flash` | API key | +| `ollama` | `llama3` | None (local) | +| Custom | Configurable | API key | + +### Configuration + +```yaml +model: + provider: openai + name: gpt-4o +``` + +Or override with environment variables: + +```bash +export FORGE_MODEL_PROVIDER=anthropic +export ANTHROPIC_API_KEY=sk-ant-... +forge run +``` + +Provider is auto-detected from available API keys if not explicitly set. Provider configuration is resolved via `ResolveModelConfig()` in priority order: + +1. **CLI flag** `--provider` (highest priority) +2. **Environment variables**: `FORGE_MODEL_PROVIDER`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY` +3. **forge.yaml** `model` section (lowest priority) + +### OpenAI OAuth + +For OpenAI, Forge supports browser-based OAuth login (matching the Codex CLI flow) as an alternative to API keys: + +```bash +forge init my-agent +# Select "OpenAI" -> "Login with browser (OAuth)" +# Browser opens for authentication +``` + +OAuth tokens are stored in `~/.forge/credentials/openai.json` and automatically refreshed. + +### Organization ID (OpenAI Enterprise) + +Enterprise OpenAI accounts can set an Organization ID to route API requests to the correct org: + +```yaml +model: + provider: openai + name: gpt-4o + organization_id: "org-xxxxxxxxxxxxxxxxxxxxxxxx" +``` + +Or via environment variable (overrides YAML): + +```bash +export OPENAI_ORG_ID=org-xxxxxxxxxxxxxxxxxxxxxxxx +``` + +The `OpenAI-Organization` header is sent on all OpenAI API requests (chat, embeddings, responses). Fallback providers inherit the primary org ID unless overridden per-fallback. The org ID is also injected into skill subprocess environments as `OPENAI_ORG_ID`. + +### Fallback Chains + +Configure fallback providers for automatic failover when the primary provider is unavailable: + +```yaml +model: + provider: openai + name: gpt-4o + fallbacks: + - provider: anthropic + name: claude-sonnet-4-20250514 + - provider: gemini +``` + +Or via environment variable: + +```bash +export FORGE_MODEL_FALLBACKS="anthropic:claude-sonnet-4-20250514,gemini:gemini-2.5-flash" +``` + +Fallback behavior: +- **Retriable errors** (rate limits, overloaded, timeouts) try the next provider +- **Non-retriable errors** (auth, billing, bad format) abort immediately +- Per-provider exponential backoff cooldowns prevent thundering herd +- Fallbacks are also auto-detected from available API keys when not explicitly configured + +## Executor Types + +The runtime supports multiple executor implementations: + +| Executor | Use Case | +|----------|----------| +| `LLMExecutor` | Custom agents with LLM-powered tool calling | +| `SubprocessExecutor` | Framework agents (CrewAI, LangChain) running as subprocesses | +| `StubExecutor` | Returns canned responses for testing | + +Executor selection happens in `runner.go` based on framework type and configuration. + +## Running Modes + +### `forge run` — Foreground Server + +Run the agent as a foreground HTTP server. Used for development and container deployments. + +```bash +# Development (all interfaces, immediate shutdown) +forge run --with slack --port 8080 + +# Container deployment +forge run --host 0.0.0.0 --shutdown-timeout 30s +``` + +| Flag | Default | Description | +|------|---------|-------------| +| `--port` | `8080` | HTTP server port | +| `--host` | `""` (all interfaces) | Bind address | +| `--shutdown-timeout` | `0` (immediate) | Graceful shutdown timeout | +| `--with` | — | Channel adapters (e.g. `slack,telegram`) | +| `--mock-tools` | `false` | Use mock executor for testing | +| `--model` | — | Override model name | +| `--provider` | — | Override LLM provider | +| `--env` | `.env` | Path to env file | +| `--enforce-guardrails` | `false` | Enforce guardrail violations as errors | + +### `forge serve` — Background Daemon + +Manage the agent as a background daemon process with PID/log management. + +```bash +# Start daemon (secure defaults: 127.0.0.1, 30s shutdown timeout) +forge serve + +# Start on custom port +forge serve start --port 9090 --host 0.0.0.0 + +# Stop the daemon +forge serve stop + +# Check status (PID, uptime, health) +forge serve status + +# View recent logs (last 100 lines) +forge serve logs +``` + +| Subcommand | Description | +|------------|-------------| +| `start` (default) | Start the daemon in background | +| `stop` | Send SIGTERM (10s timeout, SIGKILL fallback) | +| `status` | Show PID, listen address, health check | +| `logs` | Tail `.forge/serve.log` | + +The daemon forks `forge run` in the background with `setsid`, writes state to `.forge/serve.json`, and redirects output to `.forge/serve.log`. Passphrase prompting for encrypted secrets happens in the parent process (which has TTY access) before forking. + +## File Output Directory + +The runtime configures a `FilesDir` for tool-generated files (e.g., from `file_create`). This directory defaults to `/.forge/files/` and is injected into the execution context so tools can write files that other tools can reference by path. + +``` +/ + .forge/ + files/ ← file_create output (patches.yaml, reports, etc.) + sessions/ ← conversation persistence + memory/ ← long-term memory +``` + +## Conversation Memory + +For details on session persistence, context window management, compaction, and long-term memory, see the [Memory System](/docs/core-concepts/memory-system) documentation. + +## Hooks + +The engine fires hooks at key points in the loop. See [Hooks](/docs/core-concepts/hooks) for details. + +## Streaming + +The current implementation (v1) runs the full tool-calling loop non-streaming. `ExecuteStream` calls `Execute` internally and emits the final response as a single message on a channel. True word-by-word streaming during tool loops is planned for v2. diff --git a/src/content/docs/core-concepts/scheduling.md b/src/content/docs/core-concepts/scheduling.md new file mode 100644 index 0000000..c3c6bad --- /dev/null +++ b/src/content/docs/core-concepts/scheduling.md @@ -0,0 +1,59 @@ +--- +title: Cron Scheduling +description: "Schedule recurring agent tasks with cron expressions — built-in scheduler, runtime schedule tools, and channel delivery." +order: 8 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/core-concepts/scheduling.md +--- + +# Cron Scheduling + +Forge includes a built-in cron scheduler for recurring tasks, configurable in `forge.yaml` or created dynamically by the agent at runtime. + +## Configuration + +```yaml +schedules: + - id: daily-report + cron: "@daily" + task: "Generate and send the daily status report" + skill: "tavily-research" # optional: invoke a specific skill + channel: telegram # optional: deliver results to a channel + channel_target: "-100123456" # optional: destination chat/channel ID +``` + +## Cron Expressions + +| Format | Example | Description | +|--------|---------|-------------| +| 5-field standard | `*/15 * * * *` | Every 15 minutes | +| Aliases | `@hourly`, `@daily`, `@weekly`, `@monthly` | Common intervals | +| Intervals | `@every 5m`, `@every 1h30m` | Duration-based (minimum 1 minute) | + +## Schedule Tools + +The agent has four built-in tools for managing schedules at runtime: + +| Tool | Description | +|------|-------------| +| `schedule_set` | Create or update a recurring schedule | +| `schedule_list` | List all active and inactive schedules | +| `schedule_delete` | Remove a schedule (LLM-created only; YAML-defined cannot be deleted) | +| `schedule_history` | View execution history for scheduled tasks | + +Schedules can also be managed via the CLI: + +```bash +forge schedule list +``` + +## Channel Delivery + +When a schedule includes `channel` and `channel_target`, the agent's response is automatically delivered to the specified channel after each execution. When schedules are created from channel conversations (Slack, Telegram), the channel context is automatically available so the agent can capture the delivery target. + +## Execution Details + +- **Tick interval**: 30 seconds +- **Overlap prevention**: A schedule won't fire again if its previous run is still in progress +- **Persistence**: Schedules are stored in `.forge/memory/SCHEDULES.md` and survive restarts +- **History**: The last 50 executions are recorded with status, duration, and correlation IDs +- **Audit events**: `schedule_fire`, `schedule_complete`, `schedule_skip`, `schedule_modify` diff --git a/src/content/docs/core-concepts/tools-and-builtins.md b/src/content/docs/core-concepts/tools-and-builtins.md index 914fc66..9a788c5 100644 --- a/src/content/docs/core-concepts/tools-and-builtins.md +++ b/src/content/docs/core-concepts/tools-and-builtins.md @@ -13,7 +13,7 @@ This page covers every tool type: the 8 built-in tools, 3 adapter tools, skill-d ## Builtin Tools -These 8 tools are always available to every Forge agent. They cover the most common operations an LLM agent needs. +These 9 tools are always available to every Forge agent. They cover the most common operations an LLM agent needs. | Tool | Purpose | Egress Enforcement | |---|---|---| @@ -25,9 +25,19 @@ These 8 tools are always available to every Forge agent. They cover the most com | `math_calculate` | Arithmetic calculations | No | | `web_search` | Quick web lookups (Tavily or Perplexity provider) | Yes — EgressClientFromContext | | `read_skill` | Load full SKILL.md instructions on demand | No (filesystem only) | +| `file_create` | Generate downloadable files (written to disk and uploaded to channels) | No (filesystem only) | Tools that make no network calls have no egress enforcement. Tools that do (`http_request`, `web_search`) are wrapped by the egress enforcer so they can only reach allowed domains. +### file_create + +The `file_create` tool generates downloadable files that are both written to disk and uploaded to the user's channel (Slack/Telegram). Files are stored in `.forge/files/`. + +- Accepts a filename with extension and full content as text +- Returns filename, content, MIME type, and absolute disk path +- Supports common extensions: `.md`, `.json`, `.yaml`, `.py`, `.ts`, `.csv`, `.html`, etc. +- **Security**: filenames containing path separators or traversal patterns (`..`) are rejected + ## Adapter Tools Adapters bridge your agent to external tool ecosystems. Each adapter handles protocol translation and applies egress enforcement to outbound requests. @@ -68,6 +78,10 @@ These tools are only registered when specific conditions are met: | `memory_search` | Long-term memory enabled | Hybrid search over agent memory | | `memory_get` | Long-term memory enabled | Read specific memory files | | `cli_execute` | Configured or auto-derived from skills | Run allowlisted binaries | +| `schedule_set` | Scheduling configured | Create or update a recurring cron schedule | +| `schedule_list` | Scheduling configured | List all active and inactive schedules | +| `schedule_delete` | Scheduling configured | Remove an LLM-created schedule | +| `schedule_history` | Scheduling configured | View execution history for scheduled tasks | Conditional tools are not part of the `builtins.All()` set. They are added to the agent's tool set during compilation based on your configuration and the skills you have installed. @@ -75,12 +89,15 @@ Conditional tools are not part of the `builtins.All()` set. They are added to th `cli_execute` is the bridge between binary-backed skills and execution. When the LLM reads a binary-backed skill's instructions (via `read_skill`) and decides to act, it invokes `cli_execute` to run the required binary. -Key properties: +The tool implements seven security layers: -- **No shell** — uses `exec.Command` directly, not `sh -c`. This prevents shell injection. -- **Binary allowlist** — only binaries listed in `allowed_binaries` can be executed. Any attempt to run an unlisted binary is rejected. -- **Environment isolation** — only variables listed in `env_passthrough` are available to the subprocess. -- **Timeout enforcement** — each invocation has a configurable timeout. +1. **Binary allowlist** — only binaries listed in `allowed_binaries` can be executed +2. **Path resolution** — binaries are resolved to absolute paths via `exec.LookPath` at startup +3. **Argument validation** — rejects `$(`, backticks, or newlines to prevent injection +4. **Timeout enforcement** — configurable per invocation (default: 120s) +5. **No shell** — uses `exec.CommandContext` directly, not `sh -c` — no shell expansion +6. **Environment isolation** — only variables listed in `env_passthrough` are available to the subprocess +7. **Output limits** — prevents memory exhaustion (default: 1MB) Both `allowed_binaries` and `env_passthrough` are auto-derived from skill metadata. When a skill declares `metadata.forge.requires.bins: [curl]`, Forge automatically adds `curl` to the binary allowlist. When a skill declares required or optional environment variables, those are added to `env_passthrough`. You can also configure these manually in `forge.yaml`. diff --git a/src/content/docs/getting-started/configuration.md b/src/content/docs/getting-started/configuration.md index 9b82c3f..457913e 100644 --- a/src/content/docs/getting-started/configuration.md +++ b/src/content/docs/getting-started/configuration.md @@ -17,10 +17,13 @@ Here is a fully annotated `forge.yaml` showing all available sections: agent_id: my-agent version: 0.1.0 framework: forge # forge (default) | crewai | langchain — "custom" accepted as alias +registry: ghcr.io/org # Container registry (optional) +entrypoint: agent.py # Required for crewai/langchain, omit for forge model: provider: openai name: gpt-4o + organization_id: "org-xxx" # OpenAI Organization ID (enterprise, optional) fallbacks: - provider: anthropic name: claude-sonnet-4-20250514 @@ -72,6 +75,14 @@ egress: allowed_domains: - custom-api.example.com - "*.github.com" + +schedules: + - id: daily-report + cron: "@daily" + task: "Generate and send the daily status report" + skill: "tavily-research" # optional: invoke a specific skill + channel: telegram # optional: deliver results to a channel + channel_target: "-100123456" # optional: destination chat/channel ID ``` The sections below walk through each block. @@ -83,6 +94,8 @@ The sections below walk through each block. | `agent_id` | string | Unique identifier for your agent. Auto-generated from the name during `forge init`. | | `version` | string | Semantic version of your agent configuration. | | `framework` | string | Runtime framework. Defaults to `forge`. Accepts `forge`, `crewai`, `langchain`, or `custom` (alias for `forge`). | +| `registry` | string | Container registry prefix for `forge package` (e.g., `ghcr.io/org`). Optional. | +| `entrypoint` | string | Entry point script. Required for `crewai`/`langchain` frameworks, omit for `forge`. | ## Model Providers @@ -92,6 +105,7 @@ The `model` block configures your primary LLM provider and optional fallback cha model: provider: openai name: gpt-4o + organization_id: "org-xxxxxxxxxxxxxxxxxxxxxxxx" # OpenAI enterprise (optional) fallbacks: - provider: anthropic name: claude-sonnet-4-20250514 @@ -301,6 +315,29 @@ forge secret list The encrypted file is safe to commit to version control. Without `FORGE_PASSPHRASE`, the contents are unreadable. +## Schedules + +The `schedules` block configures recurring cron-based tasks. See [Scheduling](/docs/core-concepts/scheduling) for full details. + +```yaml +schedules: + - id: daily-report + cron: "@daily" + task: "Generate and send the daily status report" + skill: "tavily-research" + channel: telegram + channel_target: "-100123456" +``` + +| Field | Type | Description | +|---|---|---| +| `id` | string | Unique schedule identifier | +| `cron` | string | Cron expression (`@daily`, `*/15 * * * *`, `@every 5m`) | +| `task` | string | Task description sent to the agent | +| `skill` | string | Optional skill to invoke | +| `channel` | string | Optional channel for result delivery | +| `channel_target` | string | Destination chat/channel ID | + ## Environment Variable Overrides Several configuration values can be overridden at runtime using environment variables. This is useful for CI/CD pipelines and container deployments where you do not want to modify `forge.yaml`. @@ -308,10 +345,18 @@ Several configuration values can be overridden at runtime using environment vari | Variable | Overrides | Example | |---|---|---| | `FORGE_PASSPHRASE` | Secrets encryption passphrase | `export FORGE_PASSPHRASE="my-secret-phrase"` | +| `FORGE_MODEL_PROVIDER` | `model.provider` | `FORGE_MODEL_PROVIDER=anthropic` | | `FORGE_MEMORY_PERSISTENCE` | `memory.persistence` | `FORGE_MEMORY_PERSISTENCE=false` | | `FORGE_MEMORY_LONG_TERM` | `memory.long_term` | `FORGE_MEMORY_LONG_TERM=true` | | `FORGE_EMBEDDING_PROVIDER` | `memory.embedding_provider` | `FORGE_EMBEDDING_PROVIDER=gemini` | | `FORGE_MODEL_FALLBACKS` | `model.fallbacks` | `FORGE_MODEL_FALLBACKS="anthropic:claude-sonnet-4-20250514,gemini:gemini-2.5-flash"` | +| `OPENAI_ORG_ID` | `model.organization_id` | `OPENAI_ORG_ID=org-xxx` | +| `TAVILY_API_KEY` | Tavily web search API key | `TAVILY_API_KEY=tvly-...` | +| `PERPLEXITY_API_KEY` | Perplexity web search API key | `PERPLEXITY_API_KEY=pplx-...` | +| `WEB_SEARCH_PROVIDER` | Force web search provider | `WEB_SEARCH_PROVIDER=perplexity` | +| `OPENAI_BASE_URL` | Override OpenAI base URL | `OPENAI_BASE_URL=https://custom.api.com/v1` | +| `ANTHROPIC_BASE_URL` | Override Anthropic base URL | `ANTHROPIC_BASE_URL=https://custom.api.com` | +| `OLLAMA_BASE_URL` | Override Ollama base URL | `OLLAMA_BASE_URL=http://localhost:11434` | Environment variables take precedence over `forge.yaml` values. The original file is not modified. diff --git a/src/content/docs/getting-started/contributing.md b/src/content/docs/getting-started/contributing.md new file mode 100644 index 0000000..1a03190 --- /dev/null +++ b/src/content/docs/getting-started/contributing.md @@ -0,0 +1,156 @@ +--- +title: Contributing to Forge +description: "How to set up a development environment, add features, and submit pull requests to Forge." +order: 5 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/getting-started/contributing.md +--- + +# Contributing to Forge + +## Development Setup + +### Prerequisites + +- Go 1.25 or later +- `golangci-lint` (for linting) +- `goreleaser` (for releases, optional) + +### Clone and Build + +```bash +git clone https://github.com/initializ/forge.git +cd forge +make build +``` + +### Verify + +```bash +make vet +make test +``` + +## Code Organization + +``` +internal/cmd/ # CLI command definitions (one file per command) +internal/config/ # Configuration parsing +internal/models/ # Data structures (AgentSpec, ToolSpec, etc.) +internal/pipeline/ # Build pipeline engine +internal/build/ # Build stage implementations +internal/plugins/ # Framework plugin system +internal/runtime/ # Agent local runner +internal/runtime/llm # LLM client abstraction +internal/container/ # Container image building +internal/channels/ # Channel adapter system +internal/tools/ # Tool registry and implementations +internal/validate/ # Validation logic +pkg/a2a/ # Shared A2A protocol types +schemas/ # Embedded JSON schemas +templates/ # Go templates for code generation +testdata/ # Test fixtures +``` + +## How to Add... + +### A New Command + +1. Create `internal/cmd/yourcommand.go` +2. Define a `cobra.Command` variable +3. Register it in `internal/cmd/root.go`'s `init()` function +4. Add tests in `internal/cmd/yourcommand_test.go` + +### A New Build Stage + +1. Create `internal/build/yourstage.go` +2. Implement the `pipeline.Stage` interface (`Name()` and `Execute()`) +3. Add the stage to the pipeline in `internal/cmd/build.go` +4. Add tests in `internal/build/yourstage_test.go` + +### A New Tool + +**Builtin tool:** +1. Create `internal/tools/builtins/your_tool.go` +2. Implement the `tools.Tool` interface +3. Register in `internal/tools/builtins/register.go`'s `RegisterAll()` function + +**Custom tool (script-based):** +1. Create `tools/tool_yourname.py` (or `.ts`/`.js`) in the agent project +2. Forge discovers it automatically via `tools.DiscoverTools()` + +### A New Channel Adapter + +1. Create `internal/channels/yourplatform/yourplatform.go` +2. Implement the `channels.ChannelPlugin` interface +3. Register in `internal/cmd/channel.go`'s `createPlugin()` and `defaultRegistry()` +4. Add config generation in `generateChannelConfig()` and `generateEnvVars()` +5. Add tests + +### A New LLM Provider + +1. Create `internal/runtime/llm/providers/yourprovider.go` +2. Implement the `llm.Client` interface (`Chat()`, `ChatStream()`, `ModelID()`) +3. Add the provider to the factory in `internal/runtime/llm/providers/factory.go` +4. Add tests + +## Testing Guidelines + +### Unit Tests + +- Every package should have `*_test.go` files +- Test files use the same package name (white-box) or `_test` suffix (black-box) +- Use table-driven tests where appropriate +- Mock external dependencies (HTTP servers, file system, etc.) + +### Integration Tests + +- Use the `//go:build integration` build tag +- Run with `go test -tags=integration ./...` +- Integration tests may use test fixtures from `testdata/` +- No external services required (use `httptest` for mock servers) + +### Test Fixtures + +Test fixtures live in `testdata/` at the project root: +- `forge-valid.yaml` — Full valid forge.yaml +- `forge-minimal.yaml` — Bare-minimum valid config +- `forge-invalid.yaml` — Invalid config for error testing +- `agentspec-valid.json` — Valid AgentSpec JSON +- `agentspec-invalid.json` — Invalid AgentSpec for error testing +- `tool-schema.json` — Minimal tool input schema + +### Running Tests + +```bash +make test # All unit tests +make test-integration # Integration tests +make cover # Coverage report +go test -v ./internal/pipeline/... # Specific package +``` + +## Code Style + +- Run `go fmt` on all files +- Run `go vet` before committing +- Run `golangci-lint run ./...` for additional checks +- Keep functions focused and small +- Use meaningful variable names +- Add comments for non-obvious logic only + +## PR Process + +1. Create a feature branch from `develop` +2. Make your changes with tests +3. Ensure all checks pass: `make vet && make test && make fmt` +4. Push and open a pull request against `develop` +5. PRs require passing CI checks before merge + +## Release Process + +Releases are automated via GoReleaser: + +1. Ensure `develop` is stable and all tests pass +2. Merge `develop` into `main` +3. Tag the release: `git tag v0.1.0` +4. Push the tag: `git push origin v0.1.0` +5. GitHub Actions runs GoReleaser to build and publish binaries diff --git a/src/content/docs/reference/cli-reference.md b/src/content/docs/reference/cli-reference.md index cdbf037..9978bef 100644 --- a/src/content/docs/reference/cli-reference.md +++ b/src/content/docs/reference/cli-reference.md @@ -1,41 +1,286 @@ --- title: CLI Reference -description: Complete reference for all Forge CLI commands — core, skills, channels, secrets, keys, and security. +description: "Complete reference for all Forge CLI commands — init, build, run, serve, export, package, skills, channels, secrets, keys, schedule, tool, and ui." order: 1 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/reference/cli-reference.md --- # CLI Reference -Forge is a single binary with subcommands for every stage of the agent lifecycle. Default port for `forge serve` is **8080**. +Forge is a single binary with subcommands for every stage of the agent lifecycle. Default port for `forge run` is **8080**. -## Core Commands +## Global Flags -| Command | Description | -|---|---| -| `forge init` | Interactive wizard — walks through provider, key validation, channel, skills, egress review, passphrase, and generation | -| `forge build` | Compile skills, validate configuration, generate Dockerfile + K8s manifests + egress allowlist | -| `forge run` | Dev mode — runs the agent from the current directory in a single interactive session | -| `forge serve` | Service mode — multi-session, SSE streaming, structured logs | -| `forge serve --with slack` | Start the agent with a Slack channel connector | -| `forge serve --with telegram` | Start the agent with a Telegram channel connector | -| `forge package` | Build a container image using Docker, Podman, or Buildah | -| `forge package --prod` | Production build — rejects dev-open egress configurations | -| `forge export` | Export AgentSpec JSON for Initializ Command import | -| `forge validate` | Run schema validation, command compatibility, and requirements checks | +| Flag | Short | Default | Description | +|------|-------|---------|-------------| +| `--config` | | `forge.yaml` | Config file path | +| `--verbose` | `-v` | `false` | Enable verbose output | +| `--output-dir` | `-o` | `.` | Output directory | + +## `forge init` + +Initialize a new agent project. + +```bash +forge init [name] [flags] +``` + +| Flag | Short | Default | Description | +|------|-------|---------|-------------| +| `--name` | `-n` | | Agent name | +| `--framework` | `-f` | | Framework: `crewai`, `langchain`, or `custom` | +| `--language` | `-l` | | Language: `python`, `typescript`, or `go` | +| `--model-provider` | `-m` | | Model provider: `openai`, `anthropic`, `ollama`, or `custom` | +| `--channels` | | | Channel adapters (e.g., `slack,telegram`) | +| `--tools` | | | Builtin tools to enable (e.g., `web_search,http_request`) | +| `--skills` | | | Registry skills to include (e.g., `github,weather`) | +| `--api-key` | | | LLM provider API key | +| `--org-id` | | | OpenAI Organization ID (enterprise) | +| `--from-skills` | | | Path to a SKILL.md file for auto-configuration | +| `--non-interactive` | | `false` | Skip interactive prompts | + +```bash +# Interactive mode (default) +forge init my-agent + +# Non-interactive with all options +forge init my-agent \ + --framework langchain \ + --language python \ + --model-provider openai \ + --channels slack,telegram \ + --non-interactive + +# With builtin tools and registry skills +forge init my-agent \ + --framework custom \ + --model-provider openai \ + --tools web_search,http_request \ + --skills github \ + --api-key sk-... \ + --non-interactive + +# OpenAI enterprise with organization ID +forge init my-agent \ + --model-provider openai \ + --api-key sk-... \ + --org-id org-xxxxxxxxxxxxxxxxxxxxxxxx \ + --non-interactive +``` + +## `forge build` + +Build the agent container artifact. Runs the full 8-stage build pipeline. + +```bash +forge build [flags] +``` + +Uses global `--config` and `--output-dir` flags. Output is written to `.forge-output/` by default. + +## `forge validate` + +Validate agent spec and forge.yaml. + +```bash +forge validate [flags] +``` + +| Flag | Default | Description | +|------|---------|-------------| +| `--strict` | `false` | Treat warnings as errors | +| `--command-compat` | `false` | Check Command platform import compatibility | + +## `forge run` + +Run the agent locally with an A2A-compliant dev server. + +```bash +forge run [flags] +``` + +| Flag | Default | Description | +|------|---------|-------------| +| `--port` | `8080` | Port for the A2A dev server | +| `--host` | `""` (all interfaces) | Bind address | +| `--shutdown-timeout` | `0` (immediate) | Graceful shutdown timeout | +| `--mock-tools` | `false` | Use mock runtime instead of subprocess | +| `--enforce-guardrails` | `false` | Enforce guardrail violations as errors | +| `--model` | | Override model name | +| `--provider` | | LLM provider: `openai`, `anthropic`, `gemini`, or `ollama` | +| `--env` | `.env` | Path to .env file | +| `--with` | | Channel adapters (e.g., `slack,telegram`) | + +```bash +# Run with defaults +forge run + +# Run with mock tools on custom port +forge run --port 9090 --mock-tools + +# Run with LLM provider and channels +forge run --provider openai --model gpt-4o --with slack + +# Container deployment +forge run --host 0.0.0.0 --shutdown-timeout 30s +``` + +## `forge serve` + +Manage the agent as a background daemon process. + +```bash +forge serve [start|stop|status|logs] [flags] +``` + +| Subcommand | Description | +|------------|-------------| +| `start` (default) | Start the daemon in background | +| `stop` | Send SIGTERM (10s timeout, SIGKILL fallback) | +| `status` | Show PID, listen address, health check | +| `logs` | Tail `.forge/serve.log` | + +| Flag | Default | Description | +|------|---------|-------------| +| `--port` | `8080` | HTTP server port | +| `--host` | `127.0.0.1` | Bind address (secure default) | +| `--with` | | Channel adapters | + +```bash +# Start daemon (secure defaults) +forge serve + +# Start on custom port +forge serve start --port 9090 --host 0.0.0.0 + +# Stop the daemon +forge serve stop + +# Check status (PID, uptime, health) +forge serve status + +# View recent logs +forge serve logs +``` + +The daemon forks `forge run` in the background with `setsid`, writes state to `.forge/serve.json`, and redirects output to `.forge/serve.log`. + +## `forge export` + +Export agent spec for Command platform import. + +```bash +forge export [flags] +``` + +| Flag | Default | Description | +|------|---------|-------------| +| `--output` | `{agent_id}-forge.json` | Output file path | +| `--pretty` | `false` | Format JSON with indentation | +| `--include-schemas` | `false` | Embed tool schemas inline | +| `--simulate-import` | `false` | Print simulated import result | +| `--dev` | `false` | Include dev-category tools in export | + +```bash +# Export with defaults +forge export + +# Pretty-print with embedded schemas +forge export --pretty --include-schemas + +# Simulate Command import +forge export --simulate-import +``` + +## `forge package` + +Build a container image for the agent. + +```bash +forge package [flags] +``` + +| Flag | Default | Description | +|------|---------|-------------| +| `--push` | `false` | Push image to registry after building | +| `--platform` | | Target platform (e.g., `linux/amd64`) | +| `--no-cache` | `false` | Disable layer cache | +| `--dev` | `false` | Include dev tools in image | +| `--prod` | `false` | Production build (rejects dev tools and dev-open egress) | +| `--verify` | `false` | Smoke-test container after build | +| `--registry` | | Registry prefix (e.g., `ghcr.io/org`) | +| `--builder` | | Force builder: `docker`, `podman`, or `buildah` | +| `--skip-build` | `false` | Skip re-running forge build | +| `--with-channels` | `false` | Generate docker-compose.yaml with channel adapters | + +```bash +# Build image with auto-detected builder +forge package + +# Build and push to registry +forge package --registry ghcr.io/myorg --push + +# Production build +forge package --prod + +# Generate docker-compose with channels +forge package --with-channels +``` + +## `forge schedule` + +Manage cron schedules. + +```bash +forge schedule list +``` + +Lists all configured cron schedules (both YAML-defined and LLM-created). See [Scheduling](/docs/core-concepts/scheduling) for configuration details. + +## `forge tool` + +Manage and inspect agent tools. + +```bash +# List all available tools +forge tool list + +# Show tool details and input schema +forge tool describe +``` + +## `forge ui` + +Launch the local web dashboard. + +```bash +# Launch with defaults +forge ui + +# Specify workspace and port +forge ui --dir /path/to/workspace --port 4200 + +# Launch without auto-opening browser +forge ui --no-open +``` + +See [Web Dashboard](/docs/reference/web-dashboard) for full documentation. ## Skills Commands | Command | Description | |---|---| +| `forge skills add ` | Add a skill from the embedded registry | +| `forge skills list` | List all skills with trust levels and source tier | +| `forge skills list --category sre` | Filter by category | +| `forge skills list --tags kubernetes` | Filter by tags | | `forge skills validate` | Per-skill requirement validation (runs the autowire pipeline) | | `forge skills validate ` | Validate a single skill directory | -| `forge skills list` | List all skills with trust levels and source tier | | `forge skills trust-report ` | Show the full trust report for a specific skill | -| `forge skills autowire [--dry-run]` | Run the autowire pipeline explicitly (scan → parse → security → trust) | -| `forge skills refresh` | Re-scan local skills and re-fetch remote skills (when implemented) | +| `forge skills autowire [--dry-run]` | Run the autowire pipeline explicitly | +| `forge skills refresh` | Re-scan local skills and re-fetch remote skills | | `forge skills promote ` | Admin action: promote an `under_review` skill to `trusted` | | `forge skills block ` | Admin action: force a skill to `failed` status | -| `forge skills add ` | Add a skill from the embedded registry — copies SKILL.md + scripts, checks env/secrets, deduplicates .env | | `forge skills audit` | Run a security audit with risk scores and policy checks | | `forge skills audit --format json` | Machine-readable audit output | | `forge skills sign --key ` | Sign a skill directory with an Ed25519 key | @@ -45,10 +290,13 @@ Forge is a single binary with subcommands for every stage of the agent lifecycle | Command | Description | |---|---| -| `forge channel add slack` | Interactive Slack setup — prompts for bot token and validates it | -| `forge channel add telegram` | Interactive Telegram setup — prompts for bot token and validates it | +| `forge channel add slack` | Interactive Slack setup — prompts for tokens and validates | +| `forge channel add telegram` | Interactive Telegram setup — prompts for bot token and validates | +| `forge channel serve ` | Run a standalone channel adapter (requires `AGENT_URL` env var) | +| `forge channel list` | List available channel adapters | +| `forge channel status` | Show configured channels from `forge.yaml` | -Channels run **with** the agent via `forge serve --with `, not as separate processes. +Channels run **with** the agent via `forge run --with `, not as separate processes. ## Secret Commands @@ -106,11 +354,3 @@ forge run forge skills audit forge security egress show ``` - -## Not Real Commands - -The following are **not valid** Forge commands: - -- `forge run --bundle` — there is no bundle concept for local execution -- `forge serve --bundle` — same as above -- `forge channel slack --agent URL` — channels start WITH the agent via `--with`, not as separate processes diff --git a/src/content/docs/reference/command-integration.md b/src/content/docs/reference/command-integration.md new file mode 100644 index 0000000..6ba1272 --- /dev/null +++ b/src/content/docs/reference/command-integration.md @@ -0,0 +1,140 @@ +--- +title: Command Platform Integration +description: "Integrate Forge agents with the Command platform — compile, validate, and run agents via the forge-core Go library." +order: 7 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/reference/command-integration.md +--- + +`forge-core` (`github.com/initializ/forge/forge-core`) is a pure Go library that Command imports to compile, validate, and run Forge agents. It has zero CLI, Docker, or Kubernetes dependencies. + +## Shared Runtime Base Image Pattern + +Unlike `forge build` which generates per-agent Dockerfiles with language-specific base images, Command uses a **shared runtime base image**: + +1. **No per-agent container builds**: Command does not run `forge build` or generate Dockerfiles. Instead, it imports agents via their AgentSpec JSON. + +2. **Shared base image**: Command maintains a single runtime base image that includes the Forge agent runtime, common language runtimes, and the A2A server. + +3. **Agent loading flow**: +``` +AgentSpec JSON → forgecore.Compile() → Runtime configuration + → forgecore.NewRuntime() → LLM executor with injected tools +``` + +## Importing forge-core + +```go +import ( + forgecore "github.com/initializ/forge/forge-core" + "github.com/initializ/forge/forge-core/types" + "github.com/initializ/forge/forge-core/agentspec" + "github.com/initializ/forge/forge-core/skills" + "github.com/initializ/forge/forge-core/llm" + "github.com/initializ/forge/forge-core/runtime" + "github.com/initializ/forge/forge-core/security" + "github.com/initializ/forge/forge-core/tools" + "github.com/initializ/forge/forge-core/validate" +) +``` + +## Compile API + +```go +result, err := forgecore.Compile(forgecore.CompileRequest{ + Config: cfg, + PluginConfig: pluginCfg, + SkillEntries: skillEntries, +}) +// result.Spec — *agentspec.AgentSpec +// result.CompiledSkills — *skills.CompiledSkills +// result.EgressConfig — *security.EgressConfig +// result.Allowlist — []byte (JSON) +``` + +## Validate API + +```go +valResult := forgecore.ValidateConfig(cfg) +schemaErrs, err := forgecore.ValidateAgentSpec(jsonData) +compatResult := forgecore.ValidateCommandCompat(spec) +simResult := forgecore.SimulateImport(spec) +``` + +## Runtime API + +```go +executor := forgecore.NewRuntime(forgecore.RuntimeConfig{ + LLMClient: myLLMClient, + Tools: toolRegistry, + Hooks: hookRegistry, + SystemPrompt: "You are ...", + MaxIterations: 10, + Guardrails: guardrailEngine, + Logger: logger, +}) + +resp, err := executor.Execute(ctx, task, message) +``` + +## Override Patterns + +### Model Override + +```go +client, _ := providers.NewClient("anthropic", llm.ClientConfig{ + APIKey: os.Getenv("ANTHROPIC_API_KEY"), + Model: "claude-sonnet-4-20250514", +}) +``` + +### Tool Restriction + +```go +reg := tools.NewRegistry() +builtins.RegisterAll(reg) +filtered := reg.Filter([]string{"http_request", "json_parse"}) +``` + +### Egress Tightening + +```go +egressCfg, _ := security.Resolve( + "strict", + "allowlist", + orgAllowedDomains, + toolNames, + capabilities, +) +``` + +### Skill Gating + +```go +var approved []skills.SkillEntry +for _, entry := range allEntries { + if isApproved(entry.Name) { + approved = append(approved, entry) + } +} +result, _ := forgecore.Compile(forgecore.CompileRequest{ + Config: cfg, + SkillEntries: approved, +}) +``` + +## API Stability + +forge-core follows semantic versioning. The following are stable: + +| API | Stability | +|-----|-----------| +| `forgecore.Compile()` | Stable | +| `forgecore.ValidateConfig()` | Stable | +| `forgecore.ValidateAgentSpec()` | Stable | +| `forgecore.NewRuntime()` | Stable | +| `types.ForgeConfig` struct | Stable | +| `agentspec.AgentSpec` struct | Stable | +| `llm.Client` interface | Stable | +| `runtime.ToolExecutor` interface | Stable | +| `security.EgressConfig` struct | Stable | +| `skills.SkillEntry` struct | Stable | diff --git a/src/content/docs/reference/framework-plugins.md b/src/content/docs/reference/framework-plugins.md new file mode 100644 index 0000000..0681578 --- /dev/null +++ b/src/content/docs/reference/framework-plugins.md @@ -0,0 +1,95 @@ +--- +title: Framework Plugins +description: "Extend Forge with framework plugins — support CrewAI, LangChain, and custom frameworks in the build pipeline." +order: 6 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/reference/framework-plugins.md +--- + +Forge uses a plugin system to support multiple AI agent frameworks. Each framework plugin adapts a specific framework to the Forge build pipeline, handling project detection, configuration extraction, and A2A wrapper generation. + +## Supported Frameworks + +| Framework | Plugin | Languages | Wrapper | +|-----------|--------|-----------|---------| +| CrewAI | `crewai.Plugin` | Python | `crewai_wrapper.py` | +| LangChain | `langchain.Plugin` | Python | `langchain_wrapper.py` | +| Custom | `custom.Plugin` | Python, TypeScript, Go | None (agent is the wrapper) | + +## Plugin Interface + +Every framework plugin implements `plugins.FrameworkPlugin`: + +```go +type FrameworkPlugin interface { + Name() string + DetectProject(dir string) (bool, error) + ExtractAgentConfig(dir string) (*AgentConfig, error) + GenerateWrapper(config *AgentConfig) ([]byte, error) + RuntimeDependencies() []string +} +``` + +## How Plugins Work in the Build Pipeline + +1. **Detection** — The `FrameworkAdapterStage` checks the `framework` field in `forge.yaml`. If set, it looks up the plugin by name. Otherwise, it calls `DetectProject()` on each registered plugin to auto-detect. + +2. **Extraction** — `ExtractAgentConfig()` reads framework-specific source files and produces an `AgentConfig` struct. + +3. **Wrapper Generation** — `GenerateWrapper()` produces an A2A-compliant HTTP server wrapper that launches the framework agent. + +4. **Output** — The wrapper is written to the build output directory and referenced in the Dockerfile entrypoint. + +## Writing a Custom Plugin + +To add support for a new framework: + +1. Create a new package under `internal/plugins/yourframework/`. + +2. Implement the `FrameworkPlugin` interface: + +```go +package yourframework + +import "github.com/initializ/forge/internal/plugins" + +type Plugin struct{} + +func (p *Plugin) Name() string { return "yourframework" } + +func (p *Plugin) DetectProject(dir string) (bool, error) { + return false, nil +} + +func (p *Plugin) ExtractAgentConfig(dir string) (*plugins.AgentConfig, error) { + return &plugins.AgentConfig{ + Name: "my-agent", + Description: "Agent built with YourFramework", + }, nil +} + +func (p *Plugin) GenerateWrapper(config *plugins.AgentConfig) ([]byte, error) { + return nil, nil +} + +func (p *Plugin) RuntimeDependencies() []string { + return []string{"yourframework>=1.0"} +} +``` + +3. Register the plugin in `internal/cmd/build.go`. + +## Hook System + +Forge also has a general-purpose plugin hook system for extending the build lifecycle: + +```go +type Plugin interface { + Name() string + Version() string + Init(config map[string]any) error + Hooks() []HookPoint + Execute(ctx context.Context, hook HookPoint, data map[string]any) error +} +``` + +Available hook points: `pre-build`, `post-build`, `pre-push`, `post-push`. diff --git a/src/content/docs/reference/web-dashboard.md b/src/content/docs/reference/web-dashboard.md new file mode 100644 index 0000000..a57e165 --- /dev/null +++ b/src/content/docs/reference/web-dashboard.md @@ -0,0 +1,133 @@ +--- +title: "Web Dashboard (forge ui)" +description: "Manage agents from the browser — dashboard, interactive chat, agent creation wizard, config editor, and skill builder." +order: 5 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/reference/web-dashboard.md +--- + +Forge includes a local web dashboard for managing agents from the browser — no CLI needed after launch. + +## Launch + +```bash +# Launch the dashboard +forge ui + +# Specify workspace and port +forge ui --dir /path/to/workspace --port 4200 + +# Launch without auto-opening browser +forge ui --no-open +``` + +Opens `http://localhost:4200` with a full-featured SPA for the complete agent lifecycle. + +## Dashboard + +The main view discovers all agents in the workspace directory and shows their status in real-time via SSE (Server-Sent Events). + +| Feature | Description | +|---------|-------------| +| Agent discovery | Auto-scans workspace for `forge.yaml` files | +| Start / Stop | Start and stop agents with one click | +| Daemon processes | Agents run as background daemons via `forge serve` — they survive UI shutdown | +| Live status | Real-time state updates (stopped, starting, running, errored) | +| Passphrase unlock | Prompts for `FORGE_PASSPHRASE` when agents have encrypted secrets | +| Auto-rescan | Detects new agents after creation | +| Unified management | All agents (UI-started or CLI-started) get identical Start/Stop controls | + +### Agent Lifecycle + +The UI manages agents as daemon processes using `forge serve start` / `forge serve stop` under the hood. This means: + +- **Agents survive UI shutdown** — closing the dashboard does not kill running agents. +- **Restart detection** — restarting the UI auto-discovers running agents via `.forge/serve.json` and TCP probing. +- **Unified view** — agents started from the CLI (`forge serve start`) and agents started from the UI appear identically. + +## Interactive Chat + +Click any running agent to open a chat interface that streams responses via the A2A protocol. + +| Feature | Description | +|---------|-------------| +| Streaming responses | Real-time token streaming with progress indicators | +| Markdown rendering | Code blocks, tables, lists rendered inline | +| Session history | Browse and resume previous conversations | +| Tool call visibility | See which tools the agent invokes during execution | + +## Create Agent Wizard + +A multi-step wizard (web equivalent of `forge init`) that walks through the full agent setup: + +| Step | What it does | +|------|-------------| +| Name | Set agent name with live slug preview | +| Provider | Select LLM provider (OpenAI, Anthropic, Gemini, Ollama, Custom) | +| Model & Auth | Pick from provider-specific model lists; OpenAI supports API key or browser OAuth login, plus optional Organization ID | +| Channels | Select Slack/Telegram with inline token collection | +| Tools | Select builtin tools; web_search shows Tavily vs Perplexity provider choice | +| Skills | Browse registry skills by category with inline env var collection | +| Fallback | Select backup LLM providers with API keys for automatic failover | +| Env & Security | Add extra env vars; set passphrase for AES-256-GCM secret encryption | +| Review | Summary of all selections before creation | + +## Config Editor + +Edit `forge.yaml` for any agent with a Monaco-based YAML editor: + +| Feature | Description | +|---------|-------------| +| Syntax highlighting | YAML language support with Monaco editor | +| Live validation | Validate config against the forge schema without saving | +| Save with validation | Server-side validation before writing to disk | +| Keyboard shortcut | Cmd/Ctrl+S to save | +| Restart integration | Restart agent after config changes | + +## Skills Browser + +Browse the built-in skill registry with filtering and detail view: + +| Feature | Description | +|---------|-------------| +| Grid view | Skill cards showing name, description, category, tags | +| Category filter | Filter skills by category | +| Detail panel | Click a skill to view its full SKILL.md content | +| Env requirements | Shows required, one-of, and optional env vars per skill | + +## Skill Builder + +An AI-powered conversational tool for creating custom skills. Access it via the **Build Skill** button on any agent card. + +### How It Works + +The Skill Builder uses the agent's own LLM provider to power a chat conversation that generates valid SKILL.md files and optional helper scripts. It automatically selects a stronger code-generation model when available (e.g. `gpt-4.1` for OpenAI, `claude-opus-4-6` for Anthropic). + +### Features + +| Feature | Description | +|---------|-------------| +| Conversational design | Describe what you want in plain language; the AI generates the skill | +| Live streaming | LLM responses stream token-by-token via SSE | +| Artifact extraction | Automatically parses `skill.md` and `script:` code fences from the response | +| SKILL.md preview | Live preview panel with syntax highlighting | +| Script preview | View generated helper scripts alongside the SKILL.md | +| Validation | Server-side validation checks name format, required fields, egress domains, and uniqueness | +| One-click save | Save the validated skill directly to the agent's `skills/` directory | + +### Workflow + +1. **Open** the Skill Builder from an agent card +2. **Describe** the skill you want (e.g. "Create a skill that queries Jira issues") +3. **Iterate** — the AI asks about requirements, security constraints, and env vars +4. **Review** — inspect the generated SKILL.md and scripts in the preview panel +5. **Validate** — check for errors and warnings before saving +6. **Save** — writes `skills/{name}/SKILL.md` and `skills/{name}/scripts/` to the agent directory + +## Architecture + +The dashboard is a single Go module (`forge-ui`) embedded into the `forge` binary: + +- **Daemon-based lifecycle** — the UI delegates to `forge serve start/stop` via `exec.Command`, so agents are independent OS processes that survive UI restarts. +- **Scanner as source of truth** — reads `.forge/serve.json` and does TCP probes to detect running agents. +- **Embedded frontend** — Preact + HTM SPA with no build step required. +- **Tree-shaken Monaco** — YAML-only Monaco editor bundle (~615KB). diff --git a/src/content/docs/security/audit-logging.md b/src/content/docs/security/audit-logging.md index bf91e3d..ddfd7ea 100644 --- a/src/content/docs/security/audit-logging.md +++ b/src/content/docs/security/audit-logging.md @@ -92,16 +92,41 @@ All audit event output is protected by a `sync.Mutex`. Multiple goroutines can e Forge includes built-in guardrails that evaluate content before it reaches the LLM or the user: -| Guardrail | What It Checks | Severity | +| Guardrail | What It Checks | Detection Method | |---|---|---| -| `content_filter` | Blocked word detection (configurable) | Blocking or warning | -| `no_pii` | Email, phone, SSN via regex | Blocking or warning | -| `jailbreak_protection` | Common jailbreak phrases | Blocking or warning | +| `content_filter` | Blocked words and phrases (configurable word list) | Case-insensitive substring matching | +| `no_pii` | Email addresses, phone numbers, SSNs | Regex pattern matching | +| `jailbreak_protection` | Common jailbreak phrases and prompt injection attempts | Pattern matching against known jailbreak templates | -Guardrails are configured via `PolicyScaffold` and run in one of two modes: +### Modes -- **Enforce** — blocks the request and returns an error -- **Warn** — logs a warning audit event but allows the request to proceed +Each guardrail runs in one of two modes: + +| Mode | Behavior | Use Case | +|---|---|---| +| **Enforce** | Blocks the request and returns an error to the caller | Production environments, compliance-critical agents | +| **Warn** | Logs a warning audit event (`guardrail_check`) but allows the request to proceed | Development, monitoring, gradual rollout | + +### Configuration + +Guardrails are configured via `PolicyScaffold` and enabled with the `--enforce-guardrails` flag: + +```bash +# Enforce mode — violations are errors +forge run --enforce-guardrails + +# Warn mode (default) — violations are logged +forge run +``` + +### Audit Events + +Guardrail evaluations emit `guardrail_check` audit events: + +```json +{"event":"guardrail_check","correlation_id":"...","fields":{"guardrail":"no_pii","mode":"enforce","result":"blocked","detail":"email address detected"}} +{"event":"guardrail_check","correlation_id":"...","fields":{"guardrail":"content_filter","mode":"warn","result":"flagged","detail":"blocked word: "}} +``` ## Note on File-Based Logging diff --git a/src/content/docs/security/egress-control.md b/src/content/docs/security/egress-control.md index 14c8bb0..24d411f 100644 --- a/src/content/docs/security/egress-control.md +++ b/src/content/docs/security/egress-control.md @@ -1,6 +1,6 @@ --- title: Egress Control -description: "Control what domains your agent can reach — profiles, modes, domain resolution, and runtime enforcement." +description: "Control what domains your agent can reach — profiles, modes, domain matching, subprocess proxy, and runtime enforcement." order: 1 editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/security/egress-control.md --- @@ -9,6 +9,11 @@ editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/sec Forge controls what domains your agent can reach. Enforcement happens at both build-time and runtime, so you get a clear allowlist before you deploy and strict filtering once the agent is live. +Egress security operates at two levels: + +1. **Build time** — generates allowlist artifacts and Kubernetes NetworkPolicy manifests +2. **Runtime** — an in-process `EgressEnforcer` validates every outbound HTTP request, and a local `EgressProxy` enforces the same rules on subprocess traffic (skill scripts, `cli_execute`) + ## Profiles Profiles set the overall posture for egress control. You choose a profile in your `forge.yaml` and it determines the default mode. @@ -29,32 +34,94 @@ The mode determines how outbound requests are evaluated at runtime. | `allowlist` | Only explicitly allowed domains (exact + wildcard) | | `dev-open` | All traffic allowed (rejected by `--prod`) | +## Domain Matching + +Domain matching is handled by `DomainMatcher`, shared by both the in-process enforcer and the subprocess proxy: + +- **Exact match** — `api.openai.com` matches `api.openai.com` +- **Wildcard match** — `*.github.com` matches `api.github.com` but **not** `github.com` or `notgithub.com` +- **Port stripping** — `api.openai.com:443` is matched against `api.openai.com` +- **Case insensitive** — `API.OpenAI.COM` matches `api.openai.com` +- **Localhost bypass** — `127.0.0.1`, `::1`, and `localhost` are always allowed in all modes + ## Domain Resolution -When you run `forge build`, Forge calls `Resolve()` to merge three sources into a single deduplicated, sorted allowlist: +When you run `forge build`, Forge merges three sources into a single deduplicated, sorted allowlist: 1. **Explicit domains** from `egress.allowed_domains` in `forge.yaml` -2. **Tool-inferred domains** — skills that declare egress domains in their SKILL.md frontmatter (e.g., `web_search` adds `api.tavily.com`, `api.perplexity.ai`) -3. **Capability bundles** — capability names map to known domain sets (e.g., `slack` expands to `slack.com`, `hooks.slack.com`, `api.slack.com`) - -All three sources are merged, deduplicated, and sorted alphabetically. +2. **Tool-inferred domains** — tools map to known required domains +3. **Capability bundles** — capability names expand to known domain sets + +### Tool Domain Inference + +| Tool | Inferred Domains | +|------|-----------------| +| `web_search` / `web-search` | `api.tavily.com`, `api.perplexity.ai` | +| `github_api` | `api.github.com`, `github.com` | +| `slack_notify` | `slack.com`, `hooks.slack.com` | +| `openai_completion` | `api.openai.com` | +| `anthropic_api` | `api.anthropic.com` | +| `huggingface_api` | `api-inference.huggingface.co`, `huggingface.co` | +| `google_vertex` | `us-central1-aiplatform.googleapis.com` | +| `sendgrid_email` | `api.sendgrid.com` | +| `twilio_sms` | `api.twilio.com` | +| `aws_bedrock` | `bedrock-runtime.us-east-1.amazonaws.com` | +| `azure_openai` | `openai.azure.com` | +| `tavily_research` | `api.tavily.com` | +| `tavily_search` | `api.tavily.com` | + +### Capability Bundles + +| Capability | Domains | +|-----------|---------| +| `slack` | `slack.com`, `hooks.slack.com`, `api.slack.com` | +| `telegram` | `api.telegram.org` | ## Runtime Enforcement -The `EgressEnforcer` wraps Go's `http.RoundTripper` interface around the base transport. Every outbound HTTP request passes through it before reaching the network. +### In-Process Enforcer -Matching rules: +The `EgressEnforcer` wraps Go's `http.RoundTripper` interface around the base transport. Every outbound HTTP request from in-process Go code (builtins like `http_request`, `web_search`, LLM API calls) passes through it before reaching the network. -- **Localhost always allowed** — `127.0.0.1`, `::1`, and `localhost` are never blocked regardless of mode -- **Wildcard support** — `*.github.com` matches `api.github.com` (suffix match) but does **not** match `github.com` or `notgithub.com` -- **Port stripping** — `api.openai.com:443` is matched against `api.openai.com` -- **Case insensitive** — `API.OpenAI.COM` matches `api.openai.com` - **OnAttempt callback** — fires on every request for audit logging, whether allowed or blocked - **Blocked request error** — returns `egress blocked: domain "X" not in allowlist (mode=allowlist)` +### Subprocess Egress Proxy + +Skill scripts and `cli_execute` subprocesses bypass the Go-level enforcer because they use external tools like `curl` or `wget`. The `EgressProxy` closes this gap: + +``` +┌─────────────────────────────────────────────────────┐ +│ forge run │ +│ │ +│ In-process HTTP ──→ EgressEnforcer (RoundTripper) │ +│ │ +│ Subprocesses ──→ HTTP_PROXY ──→ EgressProxy │ +│ (curl, wget, 127.0.0.1: (validates │ +│ python, etc.) domains) │ +└─────────────────────────────────────────────────────┘ +``` + +1. Before tool registration, Forge starts a local HTTP/HTTPS forward proxy on `127.0.0.1:0` (random port) +2. `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, and `https_proxy` env vars are injected into every subprocess +3. The proxy validates each request's destination hostname against the same `DomainMatcher` +4. Allowed requests are forwarded; blocked requests receive `403 Forbidden` + +| Property | Detail | +|----------|--------| +| **Binding** | `127.0.0.1:0` — localhost only, random port, never exposed externally | +| **Lifecycle** | Per `forge run` — starts before tool registration, shuts down on context cancellation | +| **Isolation** | Multiple `forge run` instances each get their own proxy on different ports | +| **HTTPS CONNECT** | Parses host from `CONNECT host:port`, validates domain, blind-relays bytes (no MITM/decryption) | +| **Audit** | Emits `egress_allowed`/`egress_blocked` audit events with `"source": "proxy"` | + +### Container Detection + +The proxy is skipped in container environments where Kubernetes `NetworkPolicy` handles egress enforcement instead. Container detection checks for `KUBERNETES_SERVICE_HOST` or `/.dockerenv`. + ## Context Propagation -The enforcer is injected into the request context via `WithEgressClient` and retrieved with `EgressClientFromContext`. Any code that uses the context-provided HTTP client gets egress enforcement automatically. When no enforcer is present in the context, the client falls back to `http.DefaultTransport`. +The enforcer is injected into the request context via `WithEgressClient` and retrieved with `EgressClientFromContext`. Any code that uses the context-provided HTTP client gets egress enforcement automatically. ## Build-Time Outputs @@ -69,7 +136,15 @@ You can inspect the resolved allowlist at any time: forge security egress show ``` -This displays every allowed domain, its source, and the active mode. +### Production vs Development + +| Setting | Production | Development | +|---------|-----------|-------------| +| Profile | `strict` or `standard` | `permissive` | +| Mode | `deny-all` or `allowlist` | `dev-open` | +| Dev tools | Filtered out | Included | +| Network policy | Enforced | Not generated | +| Egress proxy | Active | Skipped | ## Configuration @@ -81,11 +156,24 @@ egress: mode: allowlist capabilities: - slack + - telegram allowed_domains: - custom-api.example.com - "*.github.com" ``` +## Audit Events + +Both the enforcer and proxy emit structured audit events: + +```json +{"event":"egress_allowed","domain":"api.tavily.com","mode":"allowlist"} +{"event":"egress_blocked","domain":"evil.com","mode":"allowlist"} +{"event":"egress_allowed","domain":"api.tavily.com","mode":"allowlist","source":"proxy"} +``` + +Events without `"source"` come from the in-process enforcer; events with `"source": "proxy"` come from the subprocess proxy. + ## What's Next Learn how Forge evaluates skill trustworthiness in the [Trust Model](/docs/security/trust-model). diff --git a/src/content/docs/security/overview.md b/src/content/docs/security/overview.md new file mode 100644 index 0000000..ff91836 --- /dev/null +++ b/src/content/docs/security/overview.md @@ -0,0 +1,61 @@ +--- +title: Security Architecture Overview +description: "Forge's layered security model — network posture, egress enforcement, execution sandboxing, secrets, build integrity, and guardrails." +order: 0 +editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/security/overview.md +--- + +Forge implements a **layered security model** with six primary defense mechanisms designed to make AI agents safe for enterprise environments. + +## Core Security Layers + +### 1. Network Posture + +Agents use outbound-only connections with zero inbound listeners, preventing public exposure via tunnels or webhooks. This is a fundamental architectural choice — Forge agents never accept incoming connections. + +### 2. Egress Enforcement + +Three-level validation restricts outbound traffic: + +- **In-process HTTP wrapper** validates destinations before requests leave the agent +- **Local proxy** handles subprocess traffic through a controlled gateway +- **Kubernetes NetworkPolicy** enforces pod-level restrictions in production + +See [Egress Control](/docs/security/egress-control) for full details. + +### 3. Execution Sandboxing + +External code runs in restricted environments with: + +- **Environment variable isolation** — only declared vars passed through +- **Binary allowlists** for CLI execution +- **Argument validation** blocking shell metacharacters +- **Configurable timeouts** and output limits + +### 4. Secrets Management + +AES-256-GCM encrypted secret storage with Argon2id key derivation, per-agent isolation, and a three-tier resolution hierarchy. + +See [Secret Management](/docs/security/secret-management) for details. + +### 5. Build Integrity + +Ed25519 signing and SHA-256 checksums verify supply chain artifacts before execution. + +See [Build Signing](/docs/security/build-signing) for details. + +### 6. Guardrails + +Policy engine filters inbound/outbound messages for PII, jailbreak attempts, and custom content rules in enforce or warn modes. + +See [Audit Logging](/docs/security/audit-logging) for guardrails configuration. + +## Additional Protections + +**Audit Logging** tracks all security events as structured NDJSON with correlation IDs for end-to-end tracing across tool execution, egress decisions, and LLM calls. + +**Container Security** generates deployment-ready artifacts including NetworkPolicy manifests and enforces production-mode restrictions (no dev tools, no open egress modes). + +## Trust Model + +Forge evaluates skill trustworthiness through a multi-factor scoring system. See [Trust Model](/docs/security/trust-model) for the full evaluation criteria. diff --git a/src/content/docs/skills/embedded-skills.md b/src/content/docs/skills/embedded-skills.md index b940f64..8ba88cd 100644 --- a/src/content/docs/skills/embedded-skills.md +++ b/src/content/docs/skills/embedded-skills.md @@ -1,13 +1,13 @@ --- title: Embedded Skills Reference -description: "The 6 built-in skills that ship with Forge — always available, always trusted, no installation required." +description: "The 12 built-in skills that ship with Forge — always available, always trusted, no installation required." order: 1 editUrl: https://github.com/initializ/useforge.ai/edit/main/src/content/docs/skills/embedded-skills.md --- # Embedded Skills Reference -Forge ships with 6 embedded skills compiled into the binary via `go:embed`. These skills are always trusted, available offline, and shown in the `forge init` wizard. You never need to install or configure them separately — they are part of every Forge installation. +Forge ships with 12 embedded skills compiled into the binary via `go:embed`. These skills are always trusted, available offline, and shown in the `forge init` wizard. You never need to install or configure them separately — they are part of every Forge installation. ## Embedded Skills Registry @@ -19,6 +19,12 @@ Forge ships with 6 embedded skills compiled into the binary via `go:embed`. Thes | `tavily-search` | Tavily Web Search | — | `curl`, `jq` | TAVILY_API_KEY | api.tavily.com | Script -> SkillTool | | `tavily-research` | Tavily Deep Research | — | `curl`, `jq` | TAVILY_API_KEY | api.tavily.com | Script -> SkillTool (2 tools) | | `k8s-incident-triage` | K8s Incident Triage | sre | `kubectl` | none | $K8S_API_DOMAIN | Binary -> system prompt | +| `k8s-pod-rightsizer` | K8s Pod Rightsizer | sre | `kubectl` | none | $K8S_API_DOMAIN | Binary -> system prompt | +| `code-review` | Code Review | developer | `curl`, `jq`, `git` | one_of: ANTHROPIC_API_KEY, OPENAI_API_KEY | api.anthropic.com, api.openai.com | Script -> SkillTool | +| `code-review-standards` | Code Review Standards | developer | `curl`, `jq`, `git` | one_of: ANTHROPIC_API_KEY, OPENAI_API_KEY | api.anthropic.com, api.openai.com | Script -> SkillTool | +| `code-review-github` | GitHub Code Review | developer | `curl`, `jq`, `git`, `gh` | one_of: ANTHROPIC_API_KEY, OPENAI_API_KEY; GH_TOKEN | api.anthropic.com, api.openai.com, api.github.com | Script -> SkillTool | +| `codegen-react` | React Code Generator | developer | `curl`, `jq` | one_of: ANTHROPIC_API_KEY, OPENAI_API_KEY | api.anthropic.com, api.openai.com | Script -> SkillTool | +| `codegen-html` | HTML Code Generator | developer | `curl`, `jq` | one_of: ANTHROPIC_API_KEY, OPENAI_API_KEY | api.anthropic.com, api.openai.com | Script -> SkillTool | **Registration types:** @@ -155,6 +161,53 @@ The skill includes two scripts in its `scripts/` directory: Each script corresponds to a `## Tool:` section in the SKILL.md. +## Deep Dive: code-review + +The `code-review` skill provides AI-powered code review that analyzes diffs for bugs, security issues, and improvements. + +### Variants + +| Skill | Description | +|---|---| +| `code-review` | Review local diffs and files | +| `code-review-standards` | Review with configurable coding standards | +| `code-review-github` | Review GitHub PRs directly via the `gh` CLI | + +### Features + +- **Read-only analysis** — never modifies code +- **Structured JSON output** with severity levels (critical, warning, suggestion) +- **Security scanning** — detects common vulnerabilities (injection, auth bypass, secrets in code) +- **Supports local diffs and GitHub PRs** + +### code-review-github + +The GitHub variant adds PR integration: + +```bash +forge skills add code-review-github +``` + +Requires `GH_TOKEN` for GitHub API access. Can review open PRs, comment on specific lines, and suggest changes. + +## Deep Dive: codegen-react and codegen-html + +The code generation skills produce production-ready UI components. + +### codegen-react + +Generates React components with TypeScript, hooks, and common patterns. Outputs clean, tested code following modern React conventions. + +### codegen-html + +Generates standalone HTML pages with inline CSS and JavaScript. Useful for prototyping, landing pages, and email templates. + +## Skill Builder UI + +The [Web Dashboard](/docs/reference/web-dashboard) includes a **Skill Builder** — an AI-powered conversational tool for creating custom skills. Access it via the **Build Skill** button on any agent card. + +The Skill Builder uses the agent's own LLM provider to generate valid SKILL.md files and optional helper scripts through a chat conversation. It validates the generated skill against the SKILL.md format before saving. + ## What's Next Learn how to build your own skills in [Writing Custom Skills](/docs/skills/writing-custom-skills). diff --git a/src/data/navigation.ts b/src/data/navigation.ts index 974ba5f..2e638d0 100644 --- a/src/data/navigation.ts +++ b/src/data/navigation.ts @@ -1,6 +1,7 @@ export interface NavItem { label: string; href: string; + description?: string; items?: NavItem[]; } @@ -8,28 +9,36 @@ export const docsSidebar: NavItem[] = [ { label: 'Getting Started', href: '/docs/getting-started/installation', + description: 'Install Forge, create your first agent, and configure providers.', items: [ { label: 'Installation', href: '/docs/getting-started/installation' }, { label: 'Quick Start', href: '/docs/getting-started/quick-start' }, { label: 'Your First Skill', href: '/docs/getting-started/your-first-skill' }, { label: 'Configuration', href: '/docs/getting-started/configuration' }, + { label: 'Contributing', href: '/docs/getting-started/contributing' }, ], }, { label: 'Core Concepts', href: '/docs/core-concepts/how-forge-works', + description: 'Understand the architecture, SKILL.md format, tools, channels, and runtime.', items: [ { label: 'How Forge Works', href: '/docs/core-concepts/how-forge-works' }, { label: 'SKILL.md Format', href: '/docs/core-concepts/skill-md-format' }, { label: 'Tools & Builtins', href: '/docs/core-concepts/tools-and-builtins' }, { label: 'Channels', href: '/docs/core-concepts/channels' }, { label: 'Memory System', href: '/docs/core-concepts/memory-system' }, + { label: 'Runtime Engine', href: '/docs/core-concepts/runtime-engine' }, + { label: 'Hooks', href: '/docs/core-concepts/hooks' }, + { label: 'Scheduling', href: '/docs/core-concepts/scheduling' }, ], }, { label: 'Security', - href: '/docs/security/egress-control', + href: '/docs/security/overview', + description: 'Egress control, trust evaluation, secrets, and audit logging.', items: [ + { label: 'Overview', href: '/docs/security/overview' }, { label: 'Egress Control', href: '/docs/security/egress-control' }, { label: 'Trust Model', href: '/docs/security/trust-model' }, { label: 'Secret Management', href: '/docs/security/secret-management' }, @@ -40,6 +49,7 @@ export const docsSidebar: NavItem[] = [ { label: 'Skills', href: '/docs/skills/embedded-skills', + description: 'Embedded skills, writing custom skills, and contributing.', items: [ { label: 'Embedded Skills', href: '/docs/skills/embedded-skills' }, { label: 'Writing Custom Skills', href: '/docs/skills/writing-custom-skills' }, @@ -50,6 +60,7 @@ export const docsSidebar: NavItem[] = [ { label: 'Deployment', href: '/docs/deployment/docker', + description: 'Docker, Kubernetes, production checklists, and monitoring.', items: [ { label: 'Docker', href: '/docs/deployment/docker' }, { label: 'Kubernetes', href: '/docs/deployment/kubernetes' }, @@ -60,16 +71,21 @@ export const docsSidebar: NavItem[] = [ { label: 'Reference', href: '/docs/reference/cli-reference', + description: 'CLI reference, configuration schema, and platform integration.', items: [ { label: 'CLI Reference', href: '/docs/reference/cli-reference' }, { label: 'forge.yaml Schema', href: '/docs/reference/forge-yaml-schema' }, { label: 'Environment Variables', href: '/docs/reference/environment-variables' }, { label: 'Agent Skills Compatibility', href: '/docs/reference/agent-skills-compatibility' }, + { label: 'Web Dashboard', href: '/docs/reference/web-dashboard' }, + { label: 'Framework Plugins', href: '/docs/reference/framework-plugins' }, + { label: 'Command Integration', href: '/docs/reference/command-integration' }, ], }, { label: 'FAQ', href: '/docs/faq', + description: 'Common questions about Forge.', }, ]; diff --git a/src/layouts/Base.astro b/src/layouts/Base.astro index 7b2c862..e4ec324 100644 --- a/src/layouts/Base.astro +++ b/src/layouts/Base.astro @@ -37,6 +37,15 @@ const { + + + +