diff --git a/.agents/skills/apm-integrations/SKILL.md b/.agents/skills/apm-integrations/SKILL.md new file mode 100644 index 00000000000..d416ced37fd --- /dev/null +++ b/.agents/skills/apm-integrations/SKILL.md @@ -0,0 +1,154 @@ +--- +name: apm-integrations +description: | + This skill should be used when the user asks to "add a new integration", + "instrument a library", "add instrumentation for", + "create instrumentation", "new dd-trace integration", + "add tracing for", "TracingPlugin", "DatabasePlugin", "CachePlugin", + "ClientPlugin", "ServerPlugin", "CompositePlugin", "ConsumerPlugin", + "ProducerPlugin", "addHook", "shimmer.wrap", "orchestrion", + "bindStart", "bindFinish", "startSpan", "diagnostic channel", + "runStores", "reference plugin", "example plugin", "similar integration", + or needs to build, modify, or debug the instrumentation and plugin layers + for a third-party library in dd-trace-js. +--- + +# APM Integrations + +dd-trace-js provides automatic tracing for 100+ third-party libraries. Each integration consists of two decoupled layers communicating via Node.js diagnostic channels. + +## Architecture + +``` +┌──────────────────────────┐ diagnostic channels ┌─────────────────────────┐ +│ Instrumentation │ ──────────────────────────▶ │ Plugin │ +│ datadog-instrumentations │ apm:::start │ datadog-plugin- │ +│ │ apm:::finish │ │ +│ Hooks into library │ apm:::error │ Creates spans, sets │ +│ methods, emits events │ │ tags, handles errors │ +└──────────────────────────┘ └─────────────────────────┘ +``` + +**Instrumentation** (`packages/datadog-instrumentations/src/`): +Hooks into a library's internals and publishes events with context data to named diagnostic channels. Has zero knowledge of tracing — only emits events. + +**Plugin** (`packages/datadog-plugin-/src/`): +Subscribes to diagnostic channel events and creates APM spans with service name, resource, tags, and error metadata. Extends a base class providing lifecycle management. + +Both layers are always needed for a new integration. + +## Instrumentation: Orchestrion First + +**Orchestrion is the required default for all new instrumentations.** It is an AST rewriter that automatically wraps methods via JSON configuration, with correct CJS and ESM handling built in. Orchestrion handles ESM code far more reliably than traditional shimmer-based wrapping, which struggles with ESM's static module structure. + +Config lives in `packages/datadog-instrumentations/src/helpers/rewriter/instrumentations/.js`. See [Orchestrion Reference](references/orchestrion.md) for the full config format and examples. + +### When Shimmer Is Necessary Instead + +Shimmer (`addHook` + `shimmer.wrap`) should **only** be used when orchestrion cannot handle the pattern. When using shimmer, **always include a code comment explaining why orchestrion is not viable.** Valid reasons: + +- **Dynamic method interception** — methods created at runtime or on prototype chains that orchestrion's static analysis cannot reach +- **Factory patterns** — wrapping return values of factory functions +- **Argument modification** — instrumentations that need to mutate arguments before the original call + +If none of these apply, use orchestrion. For shimmer patterns, refer to existing shimmer-based instrumentations in the codebase (e.g., `packages/datadog-instrumentations/src/pg.js`). Always try to use Orchestrion when beginning a new integration! + +## Plugin Base Classes + +Plugins extend a base class matching the library type. The base class provides automatic channel subscriptions, span lifecycle, and type-specific tags. + +``` +Plugin +├── CompositePlugin — Multiple sub-plugins (produce + consume) +├── LogPlugin — Log correlation injection (no spans) +├── WebPlugin — Base web plugin +│ └── RouterPlugin — Web frameworks with middleware +└── TracingPlugin — Base for all span-creating plugins + ├── InboundPlugin — Inbound calls + │ ├── ServerPlugin — HTTP servers + │ └── ConsumerPlugin — Message consumers (DSM) + └── OutboundPlugin — Outbound calls + ├── ProducerPlugin — Message producers (DSM) + └── ClientPlugin — HTTP/RPC clients + └── StoragePlugin — Storage systems + ├── DatabasePlugin — Database clients (DBM, db.* tags) + └── CachePlugin — Key-value caches +``` + +**Wrong base class = complex workarounds.** Always match the library type to the base class. + +## Key Concepts + +### The `ctx` Object +Context flows from instrumentation to plugin: + +- **Orchestrion**: automatically provides `ctx.arguments` (method args) and `ctx.self` (instance) +- **Shimmer**: instrumentation sets named properties (`ctx.sql`, `ctx.client`, etc.) +- **Plugin sets**: `ctx.currentStore` (span), `ctx.parentStore` (parent span) +- **On completion**: `ctx.result` or `ctx.error` + +### Channel Event Lifecycle +- `runStores()` for **start** events — establishes async context (always) +- `publish()` for **finish/error** events — notification only +- `hasSubscribers` guard — skip instrumentation when no plugin listens (performance fast path) +- When shimmer is necessary, prefer `tracingChannel` (from `dc-polyfill`) over manual channels — it provides `start/end/asyncStart/asyncEnd/error` events automatically + +### Channel Prefix Patterns +- **Orchestrion**: `tracing:orchestrion::` (set via `static prefix`) +- **Shimmer + `tracingChannel`** (preferred): `tracing:apm::` (set via `static prefix`) +- **Shimmer + manual channels** (legacy): `apm:{id}:{operation}` (default, no `static prefix` needed) + +### `bindStart` / `bindFinish` +Primary plugin methods. Base classes handle most lifecycle; often only `bindStart` is needed to create the span and set tags. + +## Reference Integrations + +**Always read 1-2 references of the same type before writing or modifying code.** + +| Library Type | Plugin | Instrumentation | Base Class | +|---|---|---|---| +| Database | `datadog-plugin-pg` | `src/pg.js` | `DatabasePlugin` | +| Cache | `datadog-plugin-redis` | `src/redis.js` | `CachePlugin` | +| HTTP client | `datadog-plugin-fetch` | `src/fetch.js` | `HttpClientPlugin` (extends `ClientPlugin`) | +| Web framework | `datadog-plugin-express` | `src/express.js` | `RouterPlugin` | +| Message queue | `datadog-plugin-kafkajs` | `src/kafkajs.js` | `Producer`/`ConsumerPlugin` | +| Orchestrion | `datadog-plugin-langchain` | `rewriter/instrumentations/langchain.js` | `TracingPlugin` | + +For the complete list by base class, see [Reference Plugins](references/reference-plugins.md). + +## Debugging + +- `DD_TRACE_DEBUG=true` to see channel activity +- Log `Object.keys(ctx)` in `bindStart` to inspect available context +- Spans missing → verify `hasSubscribers` guard; check channel names match between layers +- Context lost → ensure `runStores()` (not `publish()`) for start events +- ESM fails but CJS works → check `esmFirst: true` in hooks.js (or switch to orchestrion) + +## Implementation Workflow + +Follow these steps when creating or modifying an integration: + +1. **Investigate** — Read 1-2 reference integrations of the same type (see table above). Understand the instrumentation and plugin patterns before writing code. +2. **Implement instrumentation** — Create the instrumentation in `packages/datadog-instrumentations/src/`. Use orchestrion for instrumentation. +3. **Implement plugin** — Create the plugin in `packages/datadog-plugin-/src/`. Extend the correct base class. +4. **Register** — Add entries in `packages/dd-trace/src/plugins/index.js`, `index.d.ts`, `docs/test.ts`, `docs/API.md`, and `.github/workflows/apm-integrations.yml`. +5. **Write tests** — Add unit tests and ESM integration tests. See [Testing](references/testing.md) for templates. +6. **Run tests** — Validate with: + ```bash + # Run plugin tests (preferred CI command — handles yarn services automatically) + PLUGINS="" npm run test:plugins:ci + + # If the plugin needs external services (databases, message brokers, etc.), + # check docker-compose.yml for available service names, then: + docker compose up -d + PLUGINS="" npm run test:plugins:ci + ``` +7. **Verify** — Confirm all tests pass before marking work as complete. + +## Reference Files + +- **[New Integration Guide](references/new-integration-guide.md)** — Step-by-step guide and checklist for creating a new integration end-to-end +- **[Orchestrion Reference](references/orchestrion.md)** — JSON config format, channel naming, function kinds, plugin subscription +- **[Plugin Patterns](references/plugin-patterns.md)** — `startSpan()` API, `ctx` object details, `CompositePlugin`, channel subscriptions, code style +- **[Testing](references/testing.md)** — Unit test and ESM integration test templates +- **[Reference Plugins](references/reference-plugins.md)** — All plugins organized by base class diff --git a/.agents/skills/apm-integrations/references/new-integration-guide.md b/.agents/skills/apm-integrations/references/new-integration-guide.md new file mode 100644 index 00000000000..c63af7cafd7 --- /dev/null +++ b/.agents/skills/apm-integrations/references/new-integration-guide.md @@ -0,0 +1,296 @@ +# New Integration Guide + +Step-by-step checklist for creating a new dd-trace-js integration from scratch. + +## Prerequisites + +- Read 1-2 reference integrations of the same library type (see SKILL.md reference table) +- Determine the instrumentation approach: orchestrion (default) or shimmer (only if orchestrion cannot work — document why) +- Identify the correct plugin base class for the library type + +## Step 1: Create the Instrumentation + +### Orchestrion (Default) + +Orchestrion requires three files: + +**1. JSON config** — `packages/datadog-instrumentations/src/helpers/rewriter/instrumentations/.js`: + +```javascript +module.exports = [{ + module: { + name: '', + versionRange: '>=1.0.0', + filePath: 'dist/client.js' // file containing the target method + }, + functionQuery: { + methodName: 'query', + className: 'Client', + kind: 'Async' // Async | Callback | Sync + }, + channelName: 'Client_query' +}] +``` + +To find `filePath`, inspect the installed package to locate where the target method is defined. **Many libraries duplicate classes across separate CJS and ESM builds** (e.g., `dist/cjs/client.js` and `dist/esm/client.js`). Add a separate entry for each file path with the same `functionQuery` and `channelName` — otherwise the uninstrumented module format will silently fail. + +**2. Hooks file** — `packages/datadog-instrumentations/src/.js`: + +```javascript +'use strict' + +const { addHook, getHooks } = require('./helpers/instrument') + +for (const hook of getHooks('')) { + addHook(hook, exports => exports) +} +``` + +`getHooks` reads the orchestrion config and generates `addHook` entries automatically. This file is needed so the module hooks are registered for the rewriter to process. + +**3. hooks.js entry** — (see Register in hooks.js below) + +See [Orchestrion Reference](orchestrion.md) for the full config schema, ESQuery support, and channel naming. + +### Shimmer (Only When Orchestrion Cannot Work) + +Create `packages/datadog-instrumentations/src/.js`. Always add a comment explaining why orchestrion is not viable!!! + +**When using shimmer, prefer `tracingChannel` over manual channels.** `tracingChannel` (from `dc-polyfill` or `diagnostics_channel`) automatically provides `start`, `end`, `asyncStart`, `asyncEnd`, and `error` events — less boilerplate and consistent with how orchestrion works internally. + +**Streaming example** (the main case where shimmer is needed — intercepting emitted events on returned stream objects): + +```javascript +'use strict' + +// Shimmer required: returns a stream object whose events must be +// intercepted — orchestrion wraps method return values, not emitted events. + +const { addHook } = require('./helpers/instrument') +const { tracingChannel } = require('dc-polyfill') +const shimmer = require('../../../datadog-shimmer') + +// tracingChannel is preferred over manual channels — provides start/end/asyncStart/asyncEnd/error automatically +const ch = tracingChannel('apm::') + +addHook({ name: '', versions: ['>=1.0'] }, (moduleExports) => { + shimmer.wrap(moduleExports.prototype, 'query', function (original) { + return function wrappedQuery (...args) { + if (!ch.start.hasSubscribers) { + return original.apply(this, args) + } + + const ctx = { query: args[0], client: this } + + return ch.start.runStores(ctx, () => { + try { + const stream = original.apply(this, args) + + // Wrap emit to intercept stream lifecycle events + shimmer.wrap(stream, 'emit', function (emit) { + return function (event, arg) { + switch (event) { + case 'error': + ctx.error = arg + ch.error.publish(ctx) + break + case 'end': + ch.asyncEnd.publish(ctx) + break + } + return emit.apply(this, arguments) + } + }) + + return stream + } finally { + ch.end.publish(ctx) + } + }) + } + }) + return moduleExports +}) +``` + +For other shimmer patterns, refer to existing shimmer-based instrumentations in the codebase (e.g., `packages/datadog-instrumentations/src/pg.js`). For `tracingChannel` usage, see `packages/datadog-instrumentations/src/undici.js` or `packages/datadog-instrumentations/src/aerospike.js`. + +### Register in hooks.js + +Both orchestrion and shimmer paths require an entry in `packages/datadog-instrumentations/src/helpers/hooks.js`: + +```javascript +module.exports = { + // Orchestrion or CJS-only shimmer: + '': () => require('../'), + + // Shimmer with ESM/dual packages (orchestrion handles ESM automatically): + '': { esmFirst: true, fn: () => require('../') }, +} +``` + +## Step 2: Create the Plugin + +```bash +mkdir -p packages/datadog-plugin-/{src,test} +``` + +### Choosing the Right Base Class + +| If the library... | Use | Import | Key Features | +|---|---|---|---| +| Queries a database | `DatabasePlugin` | `../../dd-trace/src/plugins/database` | DBM comment injection, `db.*` tags | +| Caches data (Redis, Memcached) | `CachePlugin` | `../../dd-trace/src/plugins/cache` | Cache-specific tags | +| Makes HTTP/RPC requests | `ClientPlugin` | `../../dd-trace/src/plugins/client` | Peer service, distributed tracing headers | +| Handles HTTP requests | `ServerPlugin` | `../../dd-trace/src/plugins/server` | Request/response lifecycle | +| Routes requests (middleware) | `RouterPlugin` | `../../dd-trace/src/plugins/router` | Middleware span tracking, route extraction | +| Produces messages | `ProducerPlugin` | `../../dd-trace/src/plugins/producer` | DSM integration, messaging tags | +| Consumes messages | `ConsumerPlugin` | `../../dd-trace/src/plugins/consumer` | DSM integration, messaging tags | +| Has multiple operations | `CompositePlugin` | `../../dd-trace/src/plugins/composite` | Combines sub-plugins | +| Injects trace context into logs | `LogPlugin` | `../../dd-trace/src/plugins/log` | No spans, log correlation | +| None of the above | `TracingPlugin` | `../../dd-trace/src/plugins/tracing` | Generic span creation | + +**Wrong base class = complex workarounds.** If fighting the base class, the choice is probably wrong. + +### Create the Plugin File + +Create `packages/datadog-plugin-/src/index.js` (adapt base class and tags to library type): + +```javascript +'use strict' + +const DatabasePlugin = require('../../dd-trace/src/plugins/database') + +class MyPlugin extends DatabasePlugin { + static id = '' + static operation = '' + // Channel prefix determines how the plugin subscribes to instrumentation events. + // Three patterns exist — set `static prefix` explicitly based on instrumentation type: + // + // Orchestrion: static prefix = 'tracing:orchestrion::' + // Shimmer + tracingChannel: static prefix = 'tracing:apm::' + // Shimmer + manual channels: omit prefix — defaults to `apm:${id}:${operation}` + static peerServicePrecursors = ['db.name'] + + bindStart (ctx) { + // Orchestrion: use ctx.arguments and ctx.self + // Shimmer: use named properties like ctx.sql, ctx.client + const { sql, client } = ctx + + this.startSpan(this.operationName(), { + service: this.serviceName({ pluginConfig: this.config }), + resource: sql, + type: 'sql', + meta: { + component: '', + 'db.type': '' + } + }, ctx) + + return ctx.currentStore + } +} + +module.exports = MyPlugin +``` + +For `CompositePlugin` (multiple operations like produce + consume), create separate sub-plugin files in `src/`. See [Plugin Patterns](plugin-patterns.md) for the composite pattern and detailed base class examples. + +## Step 3: Register the Plugin + +Add to `packages/dd-trace/src/plugins/index.js`: + +```javascript +get '' () { return require('../../../datadog-plugin-/src') }, +``` + +If multiple npm packages map to the same plugin (e.g., `redis` and `@redis/client`), add one getter per name. + +## Step 4: Add TypeScript Definitions + +In `index.d.ts`, add to the `plugins` namespace: + +```typescript +// In the Plugins interface: +'': plugins.; + +// Add plugin interface (alphabetical order): +interface extends Instrumentation {} +// With config options: +interface extends Instrumentation { + optionName?: string | boolean; +} +``` + +## Step 5: Update docs/test.ts + +Add type-check call: + +```typescript +tracer.use(''); +tracer.use('', { optionName: 'value' }); +``` + +## Step 6: Document in docs/API.md + +Add section alphabetically: + +```markdown +
+ +This plugin automatically patches the []() module. + +| Option | Default | Description | +|--------|---------|-------------| +| `service` | | Service name override. | +``` + +## Step 7: Add CI Job + +Add to `.github/workflows/apm-integrations.yml`: + +```yaml +: + runs-on: ubuntu-latest + env: + PLUGINS: + # SERVICES: # if external services needed, plus the service configuration. should match docker-compose + steps: + - uses: actions/checkout@v4 + - uses: ./.github/actions/testagent/start + - uses: ./.github/actions/node + with: + version: ${{ matrix.node-version }} + - uses: ./.github/actions/install + - run: yarn test:plugins:ci + strategy: + matrix: + node-version: [18, 22] +``` + +Check the existing workflow for the current step format. + +## Step 8: Write Tests + +See [Testing](testing.md) for complete templates. + +**Unit tests** — `packages/datadog-plugin-/test/index.spec.js` +**ESM integration tests** — `packages/datadog-plugin-/test/integration-test/` + +```bash +# CI command (preferred) — handles dependency installation via yarn services +PLUGINS="" npm run test:plugins:ci +``` + +## Checklist + +- [ ] Instrumentation created (orchestrion JSON config + hooks file, or shimmer with justification comment) +- [ ] Registered in hooks.js (required for both orchestrion and shimmer paths) +- [ ] Plugin created with correct base class +- [ ] Plugin registered in `packages/dd-trace/src/plugins/index.js` +- [ ] TypeScript definitions added to `index.d.ts` +- [ ] Type check added to `docs/test.ts` +- [ ] Documentation added to `docs/API.md` +- [ ] CI job added to `.github/workflows/apm-integrations.yml` +- [ ] Unit tests written and passing +- [ ] ESM integration tests written and passing diff --git a/.agents/skills/apm-integrations/references/orchestrion.md b/.agents/skills/apm-integrations/references/orchestrion.md new file mode 100644 index 00000000000..17d4e108084 --- /dev/null +++ b/.agents/skills/apm-integrations/references/orchestrion.md @@ -0,0 +1,207 @@ +# Orchestrion (AST Rewriter) + +Orchestrion is the **required default** for new instrumentations. It automatically wraps methods via JSON configuration with correct CJS/ESM handling built in. Orchestrion handles ESM code far more reliably than shimmer-based wrapping because it operates at the AST level rather than trying to monkey-patch module exports. + +## Required Files + +Orchestrion integrations need three files: + +``` +packages/datadog-instrumentations/src/ +├── .js # Hooks file (registers module hooks) +└── helpers/ + ├── hooks.js # Entry pointing to .js + └── rewriter/ + ├── index.js # Main rewriter logic + └── instrumentations/ + ├── langchain.js # Reference: LangChain config + └── .js # JSON config +``` + +**Hooks file** (`packages/datadog-instrumentations/src/.js`): + +```javascript +'use strict' + +const { addHook, getHooks } = require('./helpers/instrument') + +for (const hook of getHooks('')) { + addHook(hook, exports => exports) +} +``` + +`getHooks` reads the orchestrion JSON config and generates `addHook` entries so the module hooks are registered for the rewriter to process. Without this file, the rewriter will not be triggered. + +**hooks.js entry** (`packages/datadog-instrumentations/src/helpers/hooks.js`): + +```javascript +'': () => require('../'), +``` + +## Config Schema + +Each entry in the instrumentations array: + +```javascript +{ + module: { + name: string, // npm package name (e.g. "bullmq", "@langchain/core") + versionRange: string, // semver range (e.g. ">=1.0.0") + filePath: string, // path within package (e.g. "dist/cjs/classes/queue.js") + }, + + // Option A: functionQuery (recommended) + functionQuery: { + kind: 'Async' | 'Callback' | 'Sync', // transform type (see below) + methodName: string, // class method or property method name + className?: string, // scope to a specific class + functionName?: string, // target a FunctionDeclaration (alternative to methodName) + expressionName?: string, // target a FunctionExpression/ArrowFunctionExpression + index?: number, // Callback only: argument index of the callback (-1 = last) + }, + + // Option B: astQuery (advanced, for edge cases) + astQuery?: string, // raw ESQuery selector string — bypasses functionQuery entirely + + channelName: string, // used in the diagnostic channel name +} +``` + +### functionQuery Targeting + +| Field | Targets | +|---|---| +| `methodName` + `className` | A method on a specific class | +| `methodName` alone | Any class method or object property method with that name | +| `functionName` | A `FunctionDeclaration` by name | +| `expressionName` | A `FunctionExpression` or `ArrowFunctionExpression` by name | + +### astQuery (ESQuery Selectors) + +For advanced cases where `functionQuery` fields are insufficient, use `astQuery` with a raw [ESQuery](https://github.com/estools/esquery) selector string. This is parsed via `esquery.parse()` and matched against the AST directly. Internally, `functionQuery` is converted to ESQuery selectors — `astQuery` lets you write them directly. + +### Basic Example + +```javascript +// instrumentations/.js +module.exports = [ + { + module: { + name: '', + versionRange: '>=1.0.0', + filePath: 'dist/client.js' + }, + functionQuery: { + methodName: 'query', + className: 'Client', + kind: 'Async' + }, + channelName: 'Client_query' + } +] +``` + +Multiple methods can be wrapped by adding more entries to the array. + +## Channel Name Formation + +Orchestrion channels follow this pattern: +``` +tracing:orchestrion:{module.name}:{channelName}:{event} +``` + +Example with `module.name: "@langchain/core"` and `channelName: "RunnableSequence_invoke"`: +- `tracing:orchestrion:@langchain/core:RunnableSequence_invoke:start` +- `tracing:orchestrion:@langchain/core:RunnableSequence_invoke:asyncStart` +- `tracing:orchestrion:@langchain/core:RunnableSequence_invoke:asyncEnd` +- `tracing:orchestrion:@langchain/core:RunnableSequence_invoke:end` +- `tracing:orchestrion:@langchain/core:RunnableSequence_invoke:error` + +## Function Kinds and Transforms + +Orchestrion supports three transform types, selected by the `kind` field: + +| Kind | Transform | Behavior | +|------|-----------|----------| +| `Async` | `tracePromise` | Wraps in async arrow, calls `channel.tracePromise()` — handles promise resolution/rejection | +| `Callback` | `traceCallback` | Intercepts callback at `arguments[index]` (default: last arg, i.e. `-1`), wraps it to publish `asyncStart`/`asyncEnd`/`error` events | +| `Sync` | `traceSync` | Wraps in non-async arrow, calls `channel.traceSync()` — handles synchronous return/throw. **Note:** `Sync` is the default when `kind` is omitted or unrecognized. | + +All three transforms dispatch to `traceFunction` (for standalone functions) or `traceInstanceMethod` (for class methods, including inherited ones via constructor patching). + +For `Callback` kind, use the `index` field to specify which argument is the callback (defaults to `-1`, meaning the last argument). + +## Finding the Right filePath + +1. Install the package: `npm install ` +2. Search for the method definition: + ```bash + grep -r "methodName" node_modules// + ``` +3. Use the path relative to the package root + +**IMPORTANT: Patch both CJS and ESM code paths.** Many libraries duplicate their classes across separate CJS and ESM builds (e.g., `dist/cjs/client.js` and `dist/esm/client.js`). Each file path needs its own entry in the instrumentations array with the same `functionQuery` and `channelName`. If only one is patched, the instrumentation will silently fail for the other module format. + +Common locations: +- `dist/cjs/index.js` / `dist/esm/index.js` — separate CJS/ESM builds +- `dist/index.js` — single compiled output +- `lib/client.js` — source files +- `src/index.mjs` — ESM source + +## Plugin Subscription for Orchestrion + +Set `static prefix` to match the orchestrion channel base. The `TracingPlugin` base class automatically subscribes to all events and routes them to `bindStart`, `bindFinish`, etc. + +```javascript +class MyPlugin extends TracingPlugin { + static id = '' + static prefix = 'tracing:orchestrion::Client_query' + + bindStart (ctx) { + const query = ctx.arguments?.[0] + const instance = ctx.self + + this.startSpan(this.operationName(), { + resource: query, + meta: { component: '' } + }, ctx) + + return ctx.currentStore + } +} +``` + +For integrations wrapping multiple methods, create a separate plugin class per method (each with its own `static prefix`), then combine them in a `CompositePlugin`. See langchain for this pattern. + +### The `ctx` Object in Orchestrion + +- `ctx.arguments` — the original method arguments (array) +- `ctx.self` — the `this` context of the wrapped method (instance) +- `ctx.result` — return value (on asyncEnd/end) +- `ctx.error` — thrown error (on error) +- `ctx.currentStore` — set by `startSpan` in `bindStart` + +## Common Issues + +### Wrong filePath +**Symptom**: No channel events published +**Fix**: Verify the method is actually defined in that file (not re-exported from elsewhere) + +### Case Mismatch +**Symptom**: Method not found +**Fix**: Match exact class/method name casing + +### Multiple Build Outputs +**Symptom**: Works in one context, not another +**Fix**: Check if the package has separate CJS/ESM builds with different file paths; each needs its own entry in the instrumentations array + +## Reference Implementations + +**Langchain** (canonical, multi-method): +- Config: `packages/datadog-instrumentations/src/helpers/rewriter/instrumentations/langchain.js` +- Hooks file: `packages/datadog-instrumentations/src/langchain.js` +- Plugin: `packages/datadog-plugin-langchain/src/tracing.js` + +**BullMQ** (simpler, single-package): +- Config: `packages/datadog-instrumentations/src/helpers/rewriter/instrumentations/bullmq.json` +- Hooks file: `packages/datadog-instrumentations/src/bullmq.js` diff --git a/.agents/skills/apm-integrations/references/plugin-patterns.md b/.agents/skills/apm-integrations/references/plugin-patterns.md new file mode 100644 index 00000000000..8a14a796503 --- /dev/null +++ b/.agents/skills/apm-integrations/references/plugin-patterns.md @@ -0,0 +1,195 @@ +# Plugin Patterns + +## Automatic Channel Subscriptions + +**Never manually subscribe to channels.** The `TracingPlugin` base class automatically subscribes to all events (`start`, `end`, `asyncStart`, `asyncEnd`, `error`, `finish`) and routes them to plugin methods (`bindStart`, `bindEnd`, `start`, `asyncStart`, etc.). + +The channel prefix is determined by the instrumentation type. Node.js `tracingChannel` automatically adds a `tracing:` prefix to all sub-channel names. + +| Instrumentation Type | `static prefix` | Example | +|---|---|---| +| **Orchestrion** | `'tracing:orchestrion::'` | `'tracing:orchestrion:bullmq:Queue_add'` | +| **Shimmer + `tracingChannel`** (preferred for shimmer) | `'tracing:apm::'` | `'tracing:apm:undici:fetch'` | +| **Shimmer + manual channels** (legacy) | omit — defaults to `apm:${id}:${operation}` | `apm:pg:query` | + +When using shimmer, prefer `tracingChannel` over manual channels — it provides `start/end/asyncStart/asyncEnd/error` events automatically, consistent with how orchestrion works internally. + +This means the plugin only needs to define static properties and implement `bindStart`: + +### Orchestrion Plugin (preferred) +```javascript +class MyPlugin extends TracingPlugin { + static id = '' + static prefix = 'tracing:orchestrion::Client_query' + + bindStart (ctx) { + this.startSpan(this.operationName(), { + resource: ctx.arguments?.[0], + meta: { component: '' } + }, ctx) + return ctx.currentStore + } +} +``` + +### Shimmer Plugin +```javascript +class MyPlugin extends DatabasePlugin { + static id = '' + static operation = 'query' + + bindStart (ctx) { + this.startSpan(this.operationName(), { + resource: ctx.sql, + meta: { component: '' } + }, ctx) + return ctx.currentStore + } +} +``` + +Both patterns: no manual `addSub`, `addTraceSub`, or `addBind` calls needed. The base class handles it. + +## startSpan() API + +```javascript +this.startSpan(name, options, ctx) +``` + +Options: +```javascript +{ + service: 'service-name', + resource: 'SELECT * ...', + type: 'sql', // sql, web, cache, custom + kind: 'client', // client | server | producer | consumer + meta: { // String tags + component: 'mylib', + 'db.type': 'mysql', + }, + metrics: { // Numeric tags + 'db.row_count': 42 + } +} +``` + +## The ctx Object + +### Orchestrion-Based Instrumentation +```javascript +bindStart (ctx) { + const firstArg = ctx.arguments?.[0] // method arguments + const instance = ctx.self // 'this' context + const config = ctx.self?.config +} +``` + +### Shimmer-Based Instrumentation +```javascript +bindStart (ctx) { + const { sql, client, options } = ctx // named properties set by instrumentation +} +``` + +### Common Properties (Set by Plugin) +```javascript +ctx.currentStore // { span } — set by startSpan +ctx.parentStore // { span } — parent context +ctx.result // return value (on finish) +ctx.error // thrown error (on error) +``` + +## CompositePlugin Pattern + +For integrations with multiple operations (e.g., produce + consume, or multiple orchestrion methods): + +```javascript +// src/index.js +const CompositePlugin = require('../../dd-trace/src/plugins/composite') +const ProducerPlugin = require('./producer') +const ConsumerPlugin = require('./consumer') + +class MyPlugin extends CompositePlugin { + static id = '' + + static get plugins () { + return { + producer: ProducerPlugin, + consumer: ConsumerPlugin + } + } +} + +module.exports = MyPlugin +``` + +Create separate files in `src/` for each sub-plugin. Each sub-plugin gets its own `static prefix` (orchestrion) or `static operation` (shimmer). + +For orchestrion integrations wrapping multiple methods, each method gets its own plugin class with a unique `static prefix`, then all are combined via `CompositePlugin`. See langchain for this pattern. + +## Error Handling + +Base classes handle errors automatically via `ctx.error`. Explicit handling is rarely needed: + +```javascript +// Automatic — base class reads ctx.error +// Only override for custom error logic: +error (ctx) { + const span = ctx.currentStore?.span + if (span && ctx.error) { + span.setTag('error', ctx.error) + } +} +``` + +## The finish() Guard + +If this guard exists in code, **never remove it**: +```javascript +finish (ctx) { + if (!ctx.hasOwnProperty('result') && !ctx.hasOwnProperty('error')) return + const span = ctx.currentStore?.span + if (span) { + super.finish(ctx) + } +} +``` + +Ensures spans only close when the operation actually completes. Without it, spans close prematurely. + +## Code Style + +### DO +```javascript +class MyPlugin extends DatabasePlugin { + static id = 'mylib' + static operation = 'query' + + bindStart (ctx) { + this.startSpan(this.operationName(), { + resource: ctx.sql, + meta: { component: 'mylib' } + }, ctx) + return ctx.currentStore + } +} +``` + +### DON'T +```javascript +// Over-engineered — manual subscriptions, complex channel routing +class MyPlugin extends DatabasePlugin { + constructor (...args) { + super(...args) + this.addSub('apm:mylib:query:start', ctx => this.start(ctx)) // Don't do this + this.addSub('apm:mylib:query:finish', ctx => this.finish(ctx)) // Base class handles it + } + + bindStart (ctx, channel) { + if (channel.includes('foo')) { ... } // Don't route by channel name + else if (channel.includes('bar')) { ... } + } +} +``` + +**Golden rule:** The plugin should look like production plugins. Copy from references, only change what's library-specific. diff --git a/.agents/skills/apm-integrations/references/reference-plugins.md b/.agents/skills/apm-integrations/references/reference-plugins.md new file mode 100644 index 00000000000..b1ff3740a0c --- /dev/null +++ b/.agents/skills/apm-integrations/references/reference-plugins.md @@ -0,0 +1,97 @@ +# Reference Plugins by Base Class + +**When stuck, copy from working code.** Match by library type, not library name. + +## DatabasePlugin +``` +packages/datadog-plugin-pg/ +packages/datadog-plugin-mysql/ +packages/datadog-plugin-mongodb-core/ +packages/datadog-plugin-cassandra-driver/ +packages/datadog-plugin-elasticsearch/ +``` + +## CachePlugin +``` +packages/datadog-plugin-redis/ +packages/datadog-plugin-memcached/ +packages/datadog-plugin-ioredis/ +``` + +## ClientPlugin +``` +packages/datadog-plugin-http/ +packages/datadog-plugin-fetch/ +packages/datadog-plugin-undici/ +packages/datadog-plugin-grpc/ +``` + +## ServerPlugin / RouterPlugin +``` +packages/datadog-plugin-express/ +packages/datadog-plugin-fastify/ +packages/datadog-plugin-koa/ +packages/datadog-plugin-hapi/ +``` + +## ProducerPlugin / ConsumerPlugin +``` +packages/datadog-plugin-kafkajs/ +packages/datadog-plugin-amqplib/ +packages/datadog-plugin-google-cloud-pubsub/ +``` + +## CompositePlugin +``` +packages/datadog-plugin-kafkajs/ (producer + consumer) +packages/datadog-plugin-express/ (tracing + code origin) +packages/datadog-plugin-graphql/ +``` + +## LogPlugin +``` +packages/datadog-plugin-winston/ +packages/datadog-plugin-bunyan/ +packages/datadog-plugin-pino/ +``` + +## AI/LLM Plugins +``` +packages/datadog-plugin-openai/ +packages/datadog-plugin-anthropic/ +packages/datadog-plugin-langchain/ +``` + +## Key Files to Study + +For any plugin: + +| File | Purpose | +|------|---------| +| `src/index.js` | Plugin entry, base class selection | +| `src/tracing.js` | Tracing logic (if CompositePlugin) | +| `test/index.spec.js` | Test patterns | +| `test/integration-test/` | ESM testing (if exists) | + +For instrumentation: + +| File | Purpose | +|------|---------| +| `datadog-instrumentations/src/.js` | Hook logic (shimmer) or hooks file (orchestrion) | +| `datadog-instrumentations/src/helpers/hooks.js` | Registration (both shimmer and orchestrion) | +| `rewriter/instrumentations/.js` | JSON config (orchestrion) | + +## How to Use References + +1. **Find similar plugin** — match by library type, not name +2. **Read `src/index.js`** — understand base class choice and bindStart pattern +3. **Read `test/index.spec.js`** — understand test setup and assertions +4. **Copy structure** — adapt names and ctx fields to the new library +5. **Copy test patterns** — similar libraries have similar tests + +## Golden Rules + +1. If production plugins work and yours doesn't → your structure is wrong +2. Match the base class to the library type +3. Copy before creating — don't reinvent patterns +4. Test patterns transfer between similar library types diff --git a/.agents/skills/apm-integrations/references/testing.md b/.agents/skills/apm-integrations/references/testing.md new file mode 100644 index 00000000000..d500d0d52a0 --- /dev/null +++ b/.agents/skills/apm-integrations/references/testing.md @@ -0,0 +1,209 @@ +# Testing Integrations + +## Unit Tests + +Create `packages/datadog-plugin-/test/index.spec.js`: + +```javascript +'use strict' + +const assert = require('node:assert/strict') +const agent = require('../../dd-trace/test/plugins/agent') +const { withVersions } = require('../../dd-trace/test/setup/mocha') +const { ANY_STRING } = require('../../../integration-tests/helpers') + +describe('Plugin', () => { + describe('', () => { + withVersions('', '', (version) => { + let myLib + + beforeEach(() => { + return agent.load('') + }) + + beforeEach(() => { + myLib = require(`../../../versions/@${version}`) + }) + + afterEach(() => { + return agent.close({ ritmReset: false }) + }) + + it('should create a span', async () => { + // Object-based assertion (preferred) — uses assertObjectContains internally + const expectedSpanPromise = agent.assertFirstTraceSpan({ + name: '.', + service: 'test', + type: 'sql', + resource: 'SELECT 1', + meta: { + component: '', + 'db.type': '', + }, + }) + + // trigger the instrumented operation + myLib.someOperation() + + await expectedSpanPromise + }) + + it('should create spans with callback assertion', async () => { + + // Callback-based assertion — for complex multi-span assertions + const expectedSpanPromise = agent.assertSomeTraces(traces => { + const span = traces[0][0] + assert.strictEqual(span.name, '.') + assert.strictEqual(span.meta.component, '') + }) + + // trigger the instrumented operation + myLib.someOperation() + + await expectedSpanPromise + }) + + it('should create spans if an error occurs', async () => { + // Callback-based assertion — for complex multi-span assertions + const expectedSpanPromise = agent.assertFirstTraceSpan({ + name: '.', + service: 'test', + type: 'sql', + resource: 'SELECT 1', + meta: { + component: '', + 'db.type': '', + 'error.message': '', + 'error.type': 'TypeError', + 'error.stack': ANY_STRING + }, + }) + + // trigger the instrumented operation with an error + myLib.someOperationError() + + await expectedSpanPromise + }) + }) + }) +}) +``` + +### Test Agent API + +- `agent.load(pluginNames, config, tracerConfig)` — starts test agent and loads plugin(s) +- `agent.close({ ritmReset })` — tears down agent (use `ritmReset: false` to preserve require cache) +- `agent.assertFirstTraceSpan(expectedObject)` — asserts `traces[0][0]` contains the expected properties via `assertObjectContains`. **Preferred for simple single-span assertions.** +- `agent.assertFirstTraceSpan(callback)` — runs callback with `traces[0][0]` for custom assertions +- `agent.assertSomeTraces(callback)` — runs callback with full `traces` array (array of traces, each an array of spans). Use for multi-span or multi-trace assertions. +- `agent.subscribe(handler)` — register handler called on every trace payload +- `agent.unsubscribe(handler)` — remove a subscribed handler +- `agent.reload(pluginName, config)` — reload a plugin with new config +- `agent.reset()` — clear all handlers + +### Other Test Helpers + +- `withVersions(pluginName, moduleName, cb)` — runs tests across installed versions +- `withNamingSchema(agent, ...)` — tests naming schema conventions +- `withPeerService(agent, ...)` — tests peer service tag + +## ESM Integration Tests + +ESM tests verify the plugin works with native ES module imports. They live in `packages/datadog-plugin-/test/integration-test/`. + +### server.mjs + +Minimal ESM script that initializes the tracer and triggers the instrumented operation: + +```javascript +import 'dd-trace/init.js' +import myLib from '' + +await myLib.someOperation() +``` + +### client.spec.js + +Test that spawns the ESM server and asserts spans arrive: + +```javascript +'use strict' + +const assert = require('node:assert/strict') + +const { + FakeAgent, + sandboxCwd, + useSandbox, + checkSpansForServiceName, + spawnPluginIntegrationTestProcAndExpectExit, + varySandbox, +} = require('../../../../integration-tests/helpers') +const { withVersions } = require('../../../dd-trace/test/setup/mocha') + +describe('esm', () => { + let agent + let proc + let variants + + withVersions('', '', version => { + useSandbox([`'@${version}'`], false, [ + './packages/datadog-plugin-/test/integration-test/*']) + + beforeEach(async () => { + agent = await new FakeAgent().start() + }) + + before(async function () { + variants = varySandbox('server.mjs', '', '') + }) + + afterEach(async () => { + proc && proc.kill() + await agent.stop() + }) + + for (const variant of varySandbox.VARIANTS) { + it(`is instrumented ${variant}`, async () => { + const res = agent.assertMessageReceived(({ headers, payload }) => { + assert.strictEqual(headers.host, `127.0.0.1:${agent.port}`) + assert.ok(Array.isArray(payload)) + assert.strictEqual(checkSpansForServiceName(payload, '.'), true) + }) + + proc = await spawnPluginIntegrationTestProcAndExpectExit(sandboxCwd(), variants[variant], agent.port) + + await res + }).timeout(20000) + } + }) +}) +``` + +### Key ESM Test Concepts + +- `varySandbox(filename, bindingName, namedExport, packageName, byPassDefault)` generates three import-style variants (default, star, destructure) to verify all ESM import patterns +- `varySandbox.VARIANTS` is `['default', 'star', 'destructure']` +- Pass `byPassDefault: true` as fifth argument when the module has no default export +- `useSandbox` installs package versions into a temp sandbox directory +- `spawnPluginIntegrationTestProcAndExpectExit` spawns `node