diff --git a/.claude/.gitignore b/.claude/.gitignore new file mode 100644 index 0000000000..66bb6d6462 --- /dev/null +++ b/.claude/.gitignore @@ -0,0 +1 @@ +plans diff --git a/.claude/commands/trivy-scan.md b/.claude/commands/trivy-scan.md new file mode 100644 index 0000000000..00b9ad31d4 --- /dev/null +++ b/.claude/commands/trivy-scan.md @@ -0,0 +1,171 @@ +# Trivy security scan + +Run Trivy vulnerability scans scoped to the modules modified on the current +branch, then analyse the results and provide actionable recommendations. + +## Step 1: Determine modified modules + +Run `git diff --name-only main...HEAD` to get the list of files changed on the +current branch. Map each changed file to one of the following scopes based on +its top-level directory: + +| Changed directory | Scope | +| -------------------------------------------------------------------------------------------------- | ---------------- | +| `server/` | server | +| `ui/` | ui | +| `utilities/`, `encoders/`, `terminology/`, `fhirpath/`, `library-api/`, `library-runtime/`, `lib/` | core-libraries | +| `site/` | site | +| `fhirpath-lab-api/` | fhirpath-lab-api | + +Files in other directories (e.g. `.github/`, `openspec/`, `benchmark/`, +`test-data/`, `deployment/`) do not trigger any scan scope. + +If no scopes are identified, inform the user that no scannable modules were +modified and stop. + +## Step 2: Run Trivy for each scope + +Each scope has its own `.trivyignore` file. Run Trivy from within the scope's +directory so that the local `.trivyignore` is picked up automatically. + +Common options for all scans: + +``` +--severity MEDIUM,HIGH,CRITICAL +--exit-code 0 +``` + +### Core libraries scope + +Scans from the repository root with `--skip-dirs` to exclude non-core modules. +The root `.trivyignore` contains suppressions for Spark-provided dependencies. + +Working directory: repository root. + +```bash +trivy repo . \ + --severity MEDIUM,HIGH,CRITICAL \ + --skip-files "examples/**/*,**/target/**/*,sql-on-fhir/**/*,licenses/**/*" \ + --skip-dirs "server,ui,site,fhirpath-lab-api,benchmark,test-data,deployment" \ + --exit-code 0 +``` + +### Server scope + +Scans the `server` directory. The `server/.trivyignore` contains suppressions +for Spark runtime transitive dependencies and server-specific libraries. + +Working directory: `server/`. + +```bash +cd server && trivy repo . \ + --severity MEDIUM,HIGH,CRITICAL \ + --skip-files "**/target/**/*" \ + --exit-code 0 +``` + +### UI scope + +Scans the `ui` directory. The `ui/.trivyignore` contains suppressions for +client-side JavaScript dependencies. + +Working directory: `ui/`. + +```bash +cd ui && trivy repo . \ + --severity MEDIUM,HIGH,CRITICAL \ + --exit-code 0 +``` + +### Site scope + +Scans the `site` directory. The `site/.trivyignore` contains any +site-specific suppressions. + +Working directory: `site/`. + +```bash +cd site && trivy repo . \ + --severity MEDIUM,HIGH,CRITICAL \ + --skip-files "**/target/**/*" \ + --exit-code 0 +``` + +### FHIRPath Lab API scope + +Scans the `fhirpath-lab-api` directory. The +`fhirpath-lab-api/.trivyignore` contains any API-specific suppressions. + +Working directory: `fhirpath-lab-api/`. + +```bash +cd fhirpath-lab-api && trivy repo . \ + --severity MEDIUM,HIGH,CRITICAL \ + --exit-code 0 +``` + +Run scans for different scopes in parallel where possible. Use a timeout of +5 minutes per scan. If Trivy is not installed, inform the user and suggest +`brew install trivy`. + +## Step 3: Analyse results and provide recommendations + +For each vulnerability reported by Trivy, perform a contextual impact +assessment before recommending an action. This means reading the relevant parts +of the Pathling codebase to determine whether and how the vulnerable library is +actually used. + +### Per-vulnerability analysis + +For each vulnerability: + +1. **Identify the vulnerability**: Record the CVE/GHSA ID, affected package, + installed version, fixed version (if available), and a brief description of + the attack vector. +2. **Investigate usage in our code**: Search the codebase (within the relevant + scope) to determine how the vulnerable package is used. Look for: + - Direct imports or references to the affected package or its vulnerable + classes/functions. + - Whether the vulnerable code path is reachable given our usage patterns + (e.g. do we call the affected API, use the vulnerable configuration, or + accept untrusted input that reaches the vulnerable code?). + - Whether the package is a direct dependency, a transitive dependency, or a + provided/runtime-only dependency that is not bundled in our distribution. +3. **Assess exploitability**: Based on the usage analysis, classify the + vulnerability as one of: + - **Exploitable**: The vulnerable code path is reachable in our + implementation. + - **Not exploitable**: The vulnerable code path is not reachable, or the + preconditions for exploitation do not apply (e.g. SSR-only vulnerability + in a client-side app, or a configuration we do not use). + - **Not applicable**: The package is a provided dependency not bundled in + our distribution. +4. **Recommend an action**: + - **Exploitable with fix available**: Recommend upgrading to the fixed + version. Identify the specific `pom.xml`, `package.json`, or other + dependency file that needs updating. If it is a transitive dependency, + identify the direct dependency that pulls it in and whether a version + override or exclusion is appropriate. + - **Exploitable with no fix available**: Recommend tracking for future + remediation. Suggest a workaround if one exists. + - **Not exploitable or not applicable**: Recommend adding to the + scope-specific `.trivyignore` with a comment explaining the rationale, + following the existing format in that file (comment line, then CVE/GHSA + ID). + +### Output format + +For each scope, present: + +1. **Vulnerability count** by severity (CRITICAL, HIGH, MEDIUM). +2. **Detailed findings table** with columns: CVE/GHSA ID, package, severity, + exploitability assessment, and recommended action. +3. **`.trivyignore` additions**: For vulnerabilities that should be suppressed, + provide the exact lines to add to the scope's `.trivyignore` file, following + the existing format (comment explaining rationale, then the CVE/GHSA ID). + +If no vulnerabilities are found for a scope, report that the scan passed +cleanly. + +End with an overall summary and prioritised list of actions, ordered by +exploitability and severity. diff --git a/.claude/skills/helm-charts/SKILL.md b/.claude/skills/helm-charts/SKILL.md new file mode 100755 index 0000000000..7b418a35a6 --- /dev/null +++ b/.claude/skills/helm-charts/SKILL.md @@ -0,0 +1,240 @@ +--- +name: helm-charts +description: Expert guidance for creating Helm charts with best practices. Use this skill when the user asks to create, modify, or review Helm charts, Kubernetes deployments, values.yaml files, or chart templates. Trigger keywords include "helm", "kubernetes chart", "k8s deployment", "helm template", "values.yaml". +--- + +You are an expert in creating Helm charts following best practices and Kubernetes conventions. + +## Guidelines + +- Use meaningful and descriptive names that clearly indicate purpose (avoid abbreviations). +- Follow Helm naming conventions consistently throughout the chart. +- Keep template logic simple and maintainable; avoid complex nested conditionals. +- Provide sensible defaults that work for common deployment scenarios. +- Document all configurable values comprehensively in the README.md. +- Follow Kubernetes best practices for resource definitions. + +### Naming conventions + +- Chart name: lowercase (e.g., `pathling`, `sql-on-fhir`). +- Kubernetes resource names: Use `{{ .Release.Name }}` concatenated with descriptive suffixes. + - Example: `{{ .Release.Name }}-deployment`, `{{ .Release.Name }}-service`, `{{ .Release.Name }}-pvc`. + - Do not use helper templates for naming unless specifically requested. +- Template file names: lowercase kebab-case matching the resource type. + - Example: `deployment.yaml`, `service.yaml`, `pvc.yaml`. + - Use `pvc.yaml` not `persistentvolumeclaim.yaml`. +- Value keys: camelCase for nested properties (e.g., `imagePullPolicy`, `resourceLimits`). + +### Values file structure + +- Use a single top-level key matching the chart name to namespace all chart values. +- Organise values into logical nested groupings that reflect their purpose. + - Example: `resources`, `deployment`, `config`. +- Use tilde (`~`) to explicitly indicate null or unset optional values. +- Use empty arrays (`[]`) as defaults for lists. +- Use empty objects (`{}`) as defaults for maps. +- Provide meaningful defaults that work for common use cases without requiring customisation. +- Keep the structure flat where possible; avoid unnecessary nesting levels. +- Group related configuration together (e.g., all image-related settings under an `image` key). +- **Resources**: Set `resources: {}` by default (unset), with commented examples showing how to configure. +- **Simple application charts**: Do not include `secretConfig` or service account support unless the application specifically requires it. +- **Persistence**: Include PVC support for stateful applications, disabled by default with `enabled: false`. + +Example structure for a simple application chart: + +```yaml +sqlOnFhir: + image: "sql-on-fhir-server:latest" + imagePullPolicy: "Always" + replicas: 1 + + resources: {} + # requests: + # memory: "512Mi" + # cpu: "250m" + # limits: + # memory: "1Gi" + # cpu: "500m" + + config: {} + + persistence: + enabled: false + size: "1Gi" +``` + +### Template patterns + +- Always quote string values to prevent type coercion issues. + - Example: `{{ .Values.pathling.image | quote }}`. +- **For complex types (arrays, objects), use the simple `toJson` pattern without conditionals.** + - **Correct**: `volumes: {{ toJson .Values.pathling.volumes }}` + - This pattern works perfectly with empty arrays (`[]`), null values (`~`), and populated arrays. + - Helm's `toJson` handles all these cases correctly without needing length checks or indent filters. +- **Do NOT use conditionals with `toJson` for array/object fields.** + - **Incorrect**: `{{- if gt (len .Values.pathling.volumes) 0 }}` followed by `{{ toJson .Values.pathling.volumes | indent 8 }}`. + - This anti-pattern is unnecessarily complex and error-prone. + - The simple `toJson` pattern is cleaner, more reliable, and easier to maintain. +- Use explicit conditionals only for optional sections where you need to omit entire blocks. + - Example: `{{- if gt (len .Values.pathling.config) 0 }}` when you want to completely omit the `env:` section if empty. +- Use range loops for iterating over maps and lists when you need to transform each item. + - Example: `{{- range $configKey, $configValue := .Values.pathling.config }}`. +- Separate multiple resources in a single file with `---` on its own line. +- Make resource creation conditional when appropriate. + - Example: only create secrets if `secretConfig` is defined. +- Indent template logic consistently (2 spaces is standard). +- Use `{{-` and `-}}` to control whitespace appropriately. + +Example of simple `toJson` pattern for complex types: + +```yaml +# These fields work perfectly with toJson and require no conditionals +volumes: { { toJson .Values.pathling.volumes } } +tolerations: { { toJson .Values.pathling.tolerations } } +affinity: { { toJson .Values.pathling.affinity } } +``` + +Example conditional block (for environment variables where you want to omit the entire section when empty): + +```yaml +{{- if gt (len .Values.pathling.config) 0 }} +env: + {{- range $configKey, $configValue := .Values.pathling.config }} + - name: {{ $configKey }} + value: {{ $configValue | quote }} + {{- end }} +{{- end }} +``` + +### File organisation + +- Create one template file per resource type as the default approach. + - Example: `deployment.yaml`, `service.yaml`, `configmap.yaml`. +- Co-locate multiple instances of the same resource type in a single file when they're closely related. + - Example: multiple services in `service.yaml`. +- Group dependent resources together when it improves clarity. + - Example: secrets with the deployment that consumes them. +- Use the `templates/` directory for all Kubernetes resource templates. +- Place helper functions in `templates/_helpers.tpl`. +- Keep the root directory clean with only essential files: `Chart.yaml`, `values.yaml`, `README.md`. + +### Documentation standards + +- Include a comprehensive `README.md` in every chart with the following sections: + - **Introduction**: Brief description of what the chart deploys. + - **Features**: Bullet-point list of key capabilities. + - **Prerequisites**: Required Kubernetes version, other dependencies. + - **Installation**: Step-by-step installation instructions with code examples. + - **Configuration**: Complete table of all configurable values. + - **Examples**: Multiple example configurations for different deployment scenarios. + - **Upgrading**: Notes on upgrade considerations if applicable. + - **Uninstalling**: Instructions for clean removal. +- Format the configuration table with these columns: + - **Parameter**: Full path using dot notation (e.g., `pathling.resources.limits.memory`). + - **Description**: Clear explanation of what the parameter controls. + - **Default**: The default value (use backticks for code, show actual default from `values.yaml`). +- Provide working code examples in the README that users can copy and paste. +- Use proper language identifiers in all code blocks (`yaml`, `bash`, etc.). +- Keep inline comments in `values.yaml` minimal; let the README provide detailed documentation. +- Make value names self-documenting; choose clarity over brevity. + +Example configuration table format: + +| Parameter | Description | Default | +| ---------------------------------- | ---------------------------- | -------------------------- | +| `pathling.image` | Container image to use | `pathling/pathling:latest` | +| `pathling.replicas` | Number of replicas to deploy | `1` | +| `pathling.resources.limits.memory` | Memory limit for containers | `4Gi` | + +### Configuration approach + +- Separate public configuration from sensitive configuration clearly. + - Use `config` for non-sensitive environment variables. + - Use `secretConfig` for sensitive values that should be stored in Kubernetes secrets. +- Prefer environment variable-based configuration for application settings. +- Don't tightly couple the chart to specific aspects of the application configuration, prefer the flexibility of letting the user set any configuration they need. +- Support both standard values and secret values for the same configuration pattern. +- Use consistent field naming across related configuration types. + - Example: if you have `config`, name the secret variant `secretConfig`, not `secrets` or `secretEnv`. +- Provide examples of both configuration types in the README. +- Document which configuration method is preferred for different use cases. + +Example configuration pattern: + +```yaml +pathling: + # Non-sensitive configuration + config: + PATHLING_TERMINOLOGY_SERVER_URL: "https://tx.fhir.org/r4" + PATHLING_SPARK_MASTER: "local[*]" + + # Sensitive configuration + secretConfig: + PATHLING_AUTH_TOKEN: "secret-token-value" +``` + +### Kubernetes best practices + +- Always include health probes for application containers. + - Define `startupProbe` for slow-starting applications. + - Define `livenessProbe` to detect and restart unhealthy containers. + - Define `readinessProbe` to control when containers receive traffic. +- Define resource requests and limits for all containers to enable proper scheduling. +- Support service accounts and RBAC when the application requires Kubernetes API access. +- Make image pull policies configurable (default to `Always`). +- Support pod-level configurations: + - `tolerations` for node taints. + - `affinity` rules for pod placement. + - `nodeSelector` for basic node selection. +- Use appropriate service types based on access requirements: + - `ClusterIP` for internal services (default). + - `NodePort` for external access in development. + - `LoadBalancer` for production external access. +- Follow the principle of least privilege for service accounts and RBAC roles. +- Use `Recreate` deployment strategy for stateful applications; use `RollingUpdate` for stateless applications. +- Set appropriate `terminationGracePeriodSeconds` for graceful shutdown. +- Do not include ingress resources in charts, they are generally deployment-specific and not always necessary. +- Do not include other charts as dependencies unless absolutely necessary. +- Never depend upon any chart from Bitnami. + +Example health probe configuration: + +```yaml +startupProbe: + httpGet: + path: /healthcheck + port: http + initialDelaySeconds: 30 + periodSeconds: 10 + failureThreshold: 30 + +livenessProbe: + httpGet: + path: /healthcheck + port: http + periodSeconds: 30 + failureThreshold: 3 + +readinessProbe: + httpGet: + path: /healthcheck + port: http + periodSeconds: 10 + failureThreshold: 3 +``` + +### Testing and validation + +- Use `helm template` to inspect rendered templates during development. +- Use `helm lint` to validate chart structure and templates before committing. +- Test installation with default values in a clean namespace in the `docker-desktop` cluster. +- Test with custom values that exercise different configuration paths. + +### Version management + +- Follow semantic versioning for the chart version (`major.minor.patch`). +- Increment major version for breaking changes to the values schema. +- Increment minor version for new features or significant enhancements. +- Increment patch version for bug fixes and minor improvements. +- Update `appVersion` in `Chart.yaml` to match the application version being deployed. +- Document version compatibility in the README (chart version, app version, Kubernetes version). diff --git a/.claude/skills/playwright-testing/SKILL.md b/.claude/skills/playwright-testing/SKILL.md new file mode 100644 index 0000000000..5171dc5a25 --- /dev/null +++ b/.claude/skills/playwright-testing/SKILL.md @@ -0,0 +1,206 @@ +--- +name: playwright-testing +description: Expert guidance for writing end-to-end tests with Playwright Test framework. Use this skill when writing browser automation tests, creating test suites, working with locators and assertions, mocking network requests, handling authentication, or configuring the Playwright test runner. Trigger keywords include "playwright", "e2e test", "end-to-end", "browser test", "getByRole", "locator", "toBeVisible", "page.goto", "test runner". +--- + +# Playwright Testing + +End-to-end testing framework with auto-waiting, web-first assertions, and multi-browser support. + +## Quick Start + +```typescript +import { test, expect } from "@playwright/test"; + +test("user can log in", async ({ page }) => { + await page.goto("/login"); + await page.getByLabel("Email").fill("user@example.com"); + await page.getByLabel("Password").fill("secret"); + await page.getByRole("button", { name: "Sign in" }).click(); + await expect(page.getByText("Welcome")).toBeVisible(); +}); +``` + +## Core Concepts + +### Locators (Priority Order) + +1. `page.getByRole('button', { name: 'Submit' })` — ARIA roles (most resilient) +2. `page.getByLabel('Email')` — Form labels +3. `page.getByText('Welcome')` — Visible text +4. `page.getByTestId('user-menu')` — Test IDs (explicit contracts) +5. `page.getByPlaceholder('Search')` — Placeholder text + +Avoid CSS selectors and XPath—they break with DOM changes. + +**Chaining and filtering:** + +```typescript +// Chain to narrow scope +page.locator(".modal").getByRole("button", { name: "Save" }); + +// Filter by text or child elements +page.getByRole("listitem").filter({ hasText: "Product" }); +page.getByRole("listitem").filter({ has: page.getByRole("button") }); +``` + +See [references/locators.md](references/locators.md) for complete locator API. + +### Assertions + +Auto-retrying (use these for web elements): + +```typescript +await expect(locator).toBeVisible(); +await expect(locator).toHaveText("Hello"); +await expect(locator).toHaveValue("input text"); +await expect(locator).toBeChecked(); +await expect(page).toHaveURL(/dashboard/); +``` + +Non-retrying (for static values): + +```typescript +expect(value).toBe(5); +expect(array).toContain("item"); +expect(obj).toEqual({ key: "value" }); +``` + +See [references/assertions.md](references/assertions.md) for all assertion types. + +### Actions + +```typescript +await locator.click(); +await locator.fill("text"); // Clear and type +await locator.pressSequentially("t"); // Character by character +await locator.selectOption("value"); +await locator.check(); // Checkbox +await locator.setInputFiles("file.pdf"); +await page.keyboard.press("Enter"); +``` + +All actions auto-wait for elements to be visible, stable, and enabled. + +See [references/actions.md](references/actions.md) for complete action reference. + +## Test Structure + +```typescript +import { test, expect } from "@playwright/test"; + +test.describe("Feature", () => { + test.beforeEach(async ({ page }) => { + await page.goto("/"); + }); + + test("scenario one", async ({ page }) => { + // Arrange - Act - Assert + }); + + test("scenario two", async ({ page }) => { + // ... + }); +}); +``` + +### Hooks + +- `test.beforeEach()` / `test.afterEach()` — Run before/after each test +- `test.beforeAll()` / `test.afterAll()` — Run once per worker + +## Configuration + +Minimal `playwright.config.ts`: + +```typescript +import { defineConfig, devices } from "@playwright/test"; + +export default defineConfig({ + testDir: "./tests", + fullyParallel: true, + retries: process.env.CI ? 2 : 0, + workers: process.env.CI ? 2 : undefined, + reporter: "html", + use: { + baseURL: "http://localhost:3000", + trace: "on-first-retry", + }, + projects: [ + { name: "chromium", use: { ...devices["Desktop Chrome"] } }, + { name: "firefox", use: { ...devices["Desktop Firefox"] } }, + { name: "webkit", use: { ...devices["Desktop Safari"] } }, + ], + webServer: { + command: "npm run dev", + url: "http://localhost:3000", + reuseExistingServer: !process.env.CI, + }, +}); +``` + +See [references/configuration.md](references/configuration.md) for all options. + +## CLI Commands + +```bash +npx playwright test # Run all tests +npx playwright test --ui # Interactive UI mode +npx playwright test --headed # Show browser +npx playwright test --debug # Step-through debugger +npx playwright test -g "login" # Filter by title +npx playwright test --project=chromium # Specific browser +npx playwright test --last-failed # Retry failures only +npx playwright codegen # Generate tests +npx playwright show-report # View HTML report +``` + +## Advanced Features + +### Network Mocking + +```typescript +await page.route("**/api/users", (route) => + route.fulfill({ json: [{ id: 1, name: "Mock User" }] }), +); + +await page.route("**/api/error", (route) => route.fulfill({ status: 500 })); +``` + +### Authentication State + +```typescript +// Save auth state after login +await page.context().storageState({ path: "auth.json" }); + +// Reuse in config +use: { + storageState: "auth.json"; +} +``` + +### Fixtures + +```typescript +const test = base.extend<{ userPage: Page }>({ + userPage: async ({ browser }, use) => { + const context = await browser.newContext(); + const page = await context.newPage(); + await page.goto("/login"); + // ... login + await use(page); + await context.close(); + }, +}); +``` + +See [references/advanced.md](references/advanced.md) for network mocking, auth patterns, fixtures, and Page Object Model. + +## Best Practices + +1. **Use role-based locators** — `getByRole()` is most resilient to changes +2. **Prefer auto-retrying assertions** — `await expect(locator).toBeVisible()` not `locator.isVisible()` +3. **Keep tests isolated** — Each test gets fresh browser context +4. **Avoid hardcoded waits** — Playwright auto-waits; use explicit waits only for custom conditions +5. **Test user behaviour** — Focus on what users see, not implementation details +6. **Use soft assertions sparingly** — `expect.soft()` continues after failure for comprehensive reports diff --git a/.claude/skills/playwright-testing/references/actions.md b/.claude/skills/playwright-testing/references/actions.md new file mode 100644 index 0000000000..910fcb0d2e --- /dev/null +++ b/.claude/skills/playwright-testing/references/actions.md @@ -0,0 +1,451 @@ +# Actions Reference + +## Contents + +1. [Text Input](#text-input) +2. [Click Actions](#click-actions) +3. [Form Controls](#form-controls) +4. [Keyboard](#keyboard) +5. [Mouse](#mouse) +6. [File Upload](#file-upload) +7. [Drag and Drop](#drag-and-drop) +8. [Navigation](#navigation) +9. [Waiting](#waiting) +10. [Frames](#frames) + +## Text Input + +### fill() + +Clear field and enter text: + +```typescript +await page.getByLabel("Email").fill("user@example.com"); +await page.getByRole("textbox").fill(""); // Clear field +``` + +### pressSequentially() + +Type character by character (triggers key events): + +```typescript +await page.getByLabel("Search").pressSequentially("query", { delay: 100 }); +``` + +### clear() + +Clear input field: + +```typescript +await page.getByLabel("Name").clear(); +``` + +## Click Actions + +### Basic Click + +```typescript +await page.getByRole("button", { name: "Submit" }).click(); +``` + +### Double Click + +```typescript +await page.getByText("Edit").dblclick(); +``` + +### Right Click + +```typescript +await page.getByText("Item").click({ button: "right" }); +``` + +### Click with Modifiers + +```typescript +await page.getByRole("link").click({ modifiers: ["Shift"] }); +await page.getByRole("link").click({ modifiers: ["Control"] }); // Ctrl+Click +await page.getByRole("link").click({ modifiers: ["Meta"] }); // Cmd+Click (Mac) +await page.getByRole("link").click({ modifiers: ["Alt"] }); +``` + +### Click at Position + +```typescript +await page.getByRole("canvas").click({ position: { x: 100, y: 50 } }); +``` + +### Force Click + +Bypass actionability checks: + +```typescript +await page.getByRole("button").click({ force: true }); +``` + +### Click Count + +```typescript +await page.getByText("Word").click({ clickCount: 3 }); // Triple-click to select +``` + +## Form Controls + +### Checkbox + +```typescript +await page.getByLabel("Subscribe").check(); +await page.getByLabel("Subscribe").uncheck(); +await page.getByLabel("Subscribe").setChecked(true); +await page.getByLabel("Subscribe").setChecked(false); +``` + +### Radio Button + +```typescript +await page.getByLabel("Option A").check(); +``` + +### Select Dropdown + +```typescript +// By value +await page.getByLabel("Country").selectOption("au"); + +// By label text +await page.getByLabel("Country").selectOption({ label: "Australia" }); + +// Multiple selection +await page.getByLabel("Colors").selectOption(["red", "blue"]); +``` + +### Focus + +```typescript +await page.getByLabel("Email").focus(); +``` + +## Keyboard + +### Press Key + +```typescript +await page.keyboard.press("Enter"); +await page.keyboard.press("Tab"); +await page.keyboard.press("Escape"); +await page.keyboard.press("ArrowDown"); +await page.keyboard.press("Backspace"); +``` + +### Key Combinations + +```typescript +await page.keyboard.press("Control+a"); // Select all +await page.keyboard.press("Control+c"); // Copy +await page.keyboard.press("Control+v"); // Paste +await page.keyboard.press("Meta+s"); // Cmd+S (Mac) +await page.keyboard.press("Shift+Tab"); +``` + +### Type Text + +```typescript +await page.keyboard.type("Hello World"); +await page.keyboard.type("slow typing", { delay: 100 }); +``` + +### Key Down/Up + +```typescript +await page.keyboard.down("Shift"); +await page.keyboard.press("ArrowDown"); +await page.keyboard.press("ArrowDown"); +await page.keyboard.up("Shift"); +``` + +### On Locator + +```typescript +await page.getByRole("textbox").press("Enter"); +await page.getByRole("textbox").press("Control+a"); +``` + +## Mouse + +### Move + +```typescript +await page.mouse.move(100, 200); +``` + +### Click at Coordinates + +```typescript +await page.mouse.click(100, 200); +await page.mouse.click(100, 200, { button: "right" }); +await page.mouse.dblclick(100, 200); +``` + +### Button Press/Release + +```typescript +await page.mouse.down(); +await page.mouse.move(200, 300); +await page.mouse.up(); +``` + +### Scroll + +```typescript +await page.mouse.wheel(0, 500); // Scroll down +await page.mouse.wheel(0, -500); // Scroll up +``` + +### Hover + +```typescript +await page.getByRole("menuitem").hover(); +await page.getByText("Tooltip trigger").hover(); +``` + +## File Upload + +### Single File + +```typescript +await page.getByLabel("Upload").setInputFiles("path/to/file.pdf"); +``` + +### Multiple Files + +```typescript +await page.getByLabel("Upload").setInputFiles(["file1.pdf", "file2.pdf"]); +``` + +### Directory + +```typescript +await page.getByLabel("Upload").setInputFiles("path/to/directory"); +``` + +### Buffer (In-Memory) + +```typescript +await page.getByLabel("Upload").setInputFiles({ + name: "test.txt", + mimeType: "text/plain", + buffer: Buffer.from("file content"), +}); +``` + +### Clear Selection + +```typescript +await page.getByLabel("Upload").setInputFiles([]); +``` + +### File Chooser Event + +```typescript +const fileChooserPromise = page.waitForEvent("filechooser"); +await page.getByRole("button", { name: "Upload" }).click(); +const fileChooser = await fileChooserPromise; +await fileChooser.setFiles("path/to/file.pdf"); +``` + +## Drag and Drop + +### Simple Drag + +```typescript +await page.getByText("Drag me").dragTo(page.getByText("Drop here")); +``` + +### Manual Drag + +```typescript +await page.locator("#source").hover(); +await page.mouse.down(); +await page.locator("#target").hover(); +await page.mouse.up(); +``` + +### With Coordinates + +```typescript +const source = page.locator("#source"); +const target = page.locator("#target"); + +const sourceBox = await source.boundingBox(); +const targetBox = await target.boundingBox(); + +await page.mouse.move( + sourceBox.x + sourceBox.width / 2, + sourceBox.y + sourceBox.height / 2, +); +await page.mouse.down(); +await page.mouse.move( + targetBox.x + targetBox.width / 2, + targetBox.y + targetBox.height / 2, +); +await page.mouse.up(); +``` + +## Navigation + +### Go To URL + +```typescript +await page.goto("https://example.com"); +await page.goto("/relative/path"); // Uses baseURL +await page.goto("https://example.com", { waitUntil: "networkidle" }); +``` + +Wait options: + +- `'load'` — Wait for load event (default) +- `'domcontentloaded'` — Wait for DOMContentLoaded +- `'networkidle'` — Wait until no network requests for 500ms +- `'commit'` — Wait for response received + +### Reload + +```typescript +await page.reload(); +await page.reload({ waitUntil: "networkidle" }); +``` + +### History Navigation + +```typescript +await page.goBack(); +await page.goForward(); +``` + +### Get Current URL/Title + +```typescript +const url = page.url(); +const title = await page.title(); +``` + +## Waiting + +### Wait for URL + +```typescript +await page.waitForURL("**/dashboard"); +await page.waitForURL(/\/dashboard$/); +``` + +### Wait for Load State + +```typescript +await page.waitForLoadState("load"); +await page.waitForLoadState("domcontentloaded"); +await page.waitForLoadState("networkidle"); +``` + +### Wait for Locator State + +```typescript +await page.getByRole("dialog").waitFor(); +await page.getByRole("dialog").waitFor({ state: "visible" }); +await page.getByRole("dialog").waitFor({ state: "hidden" }); +await page.getByRole("dialog").waitFor({ state: "attached" }); +await page.getByRole("dialog").waitFor({ state: "detached" }); +``` + +### Wait for Response + +```typescript +const responsePromise = page.waitForResponse("**/api/data"); +await page.getByRole("button", { name: "Load" }).click(); +const response = await responsePromise; +``` + +### Wait for Request + +```typescript +const requestPromise = page.waitForRequest("**/api/submit"); +await page.getByRole("button", { name: "Submit" }).click(); +const request = await requestPromise; +``` + +### Wait for Function + +```typescript +await page.waitForFunction(() => window.dataLoaded === true); +await page.waitForFunction(([a, b]) => a + b === 10, [3, 7]); +``` + +### Wait for Timeout + +Use sparingly—prefer explicit waits: + +```typescript +await page.waitForTimeout(1000); // 1 second +``` + +## Frames + +### Access Frame + +```typescript +// By name or URL +const frame = page.frame({ name: "myframe" }); +const frame = page.frame({ url: /example\.com/ }); + +// By frame locator +const frameLocator = page.frameLocator('iframe[name="editor"]'); +await frameLocator.getByRole("button").click(); +``` + +### Frame Locator + +```typescript +const frame = page.frameLocator("#iframe"); +await frame.getByLabel("Email").fill("user@example.com"); +await frame.getByRole("button", { name: "Submit" }).click(); +``` + +### Nested Frames + +```typescript +const outerFrame = page.frameLocator("#outer"); +const innerFrame = outerFrame.frameLocator("#inner"); +await innerFrame.getByRole("button").click(); +``` + +## Auto-Waiting + +All actions auto-wait for: + +1. **Visibility** — Element is visible +2. **Stability** — Element position is stable (not animating) +3. **Enabled** — Element is not disabled +4. **Receivable** — Element can receive events (not obscured) + +Override with `force: true` when needed: + +```typescript +await locator.click({ force: true }); +await locator.fill("text", { force: true }); +``` + +## Actionability Timeout + +Default action timeout: 30 seconds. Configure globally: + +```typescript +// playwright.config.ts +use: { + actionTimeout: 10000, // 10 seconds +} +``` + +Or per-action: + +```typescript +await locator.click({ timeout: 5000 }); +``` diff --git a/.claude/skills/playwright-testing/references/advanced.md b/.claude/skills/playwright-testing/references/advanced.md new file mode 100644 index 0000000000..ba2cc4de8b --- /dev/null +++ b/.claude/skills/playwright-testing/references/advanced.md @@ -0,0 +1,627 @@ +# Advanced Features Reference + +## Contents + +1. [Network Mocking](#network-mocking) +2. [API Testing](#api-testing) +3. [Authentication](#authentication) +4. [Fixtures](#fixtures) +5. [Page Object Model](#page-object-model) +6. [Multiple Pages and Contexts](#multiple-pages-and-contexts) +7. [Downloads](#downloads) +8. [Dialogs](#dialogs) + +## Network Mocking + +### Route Requests + +```typescript +// Mock API response +await page.route("**/api/users", (route) => + route.fulfill({ + status: 200, + contentType: "application/json", + body: JSON.stringify([{ id: 1, name: "Mock User" }]), + }), +); + +// Shorthand for JSON +await page.route("**/api/users", (route) => + route.fulfill({ json: [{ id: 1, name: "Mock User" }] }), +); +``` + +### Modify Requests + +```typescript +await page.route("**/api/**", (route) => { + const headers = { + ...route.request().headers(), + "X-Custom-Header": "value", + }; + route.continue({ headers }); +}); +``` + +### Modify Responses + +```typescript +await page.route("**/api/users", async (route) => { + const response = await route.fetch(); + const json = await response.json(); + json.push({ id: 99, name: "Injected User" }); + route.fulfill({ json }); +}); +``` + +### Abort Requests + +```typescript +// Block images +await page.route("**/*.{png,jpg,jpeg,gif}", (route) => route.abort()); + +// Block by resource type +await page.route("**/*", (route) => { + if (route.request().resourceType() === "image") { + route.abort(); + } else { + route.continue(); + } +}); +``` + +### Error Responses + +```typescript +await page.route("**/api/submit", (route) => + route.fulfill({ + status: 500, + body: JSON.stringify({ error: "Server error" }), + }), +); + +await page.route("**/api/timeout", (route) => route.abort("timedout")); +``` + +### Wait for Responses + +```typescript +const responsePromise = page.waitForResponse("**/api/data"); +await page.getByRole("button", { name: "Load" }).click(); +const response = await responsePromise; + +expect(response.status()).toBe(200); +const data = await response.json(); +``` + +### HAR Recording + +```typescript +// Record +await page.routeFromHAR("recordings/api.har", { update: true }); +await page.goto("/dashboard"); +// ... interactions recorded + +// Replay +await page.routeFromHAR("recordings/api.har"); +``` + +### Context-Level Routes + +```typescript +// Apply to all pages in context +await context.route("**/api/**", (route) => + route.fulfill({ json: { mocked: true } }), +); +``` + +### Remove Routes + +```typescript +await page.unroute("**/api/users"); +await page.unrouteAll(); +``` + +## API Testing + +### Basic Requests + +```typescript +test("API test", async ({ request }) => { + // GET + const response = await request.get("/api/users"); + expect(response.ok()).toBeTruthy(); + const users = await response.json(); + + // POST + const createResponse = await request.post("/api/users", { + data: { name: "New User", email: "new@example.com" }, + }); + expect(createResponse.status()).toBe(201); + + // PUT + await request.put("/api/users/1", { + data: { name: "Updated" }, + }); + + // DELETE + await request.delete("/api/users/1"); +}); +``` + +### Request Options + +```typescript +const response = await request.post("/api/data", { + data: { key: "value" }, // JSON body + form: { field: "value" }, // Form data + multipart: { + // Multipart form + file: { + name: "file.txt", + mimeType: "text/plain", + buffer: Buffer.from("content"), + }, + }, + headers: { "X-Custom": "header" }, + params: { page: "1" }, // Query params + timeout: 5000, + failOnStatusCode: true, // Throw on 4xx/5xx +}); +``` + +### Setup via API + +```typescript +test.beforeAll(async ({ request }) => { + // Create test data + await request.post("/api/users", { + data: { name: "Test User" }, + }); +}); + +test.afterAll(async ({ request }) => { + // Cleanup + await request.delete("/api/test-data"); +}); +``` + +### Standalone API Context + +```typescript +import { request } from "@playwright/test"; + +test("standalone API", async () => { + const apiContext = await request.newContext({ + baseURL: "https://api.example.com", + extraHTTPHeaders: { + Authorization: `Bearer ${process.env.API_TOKEN}`, + }, + }); + + const response = await apiContext.get("/data"); + await apiContext.dispose(); +}); +``` + +## Authentication + +### Save Storage State + +```typescript +// After login +await page.context().storageState({ path: "auth.json" }); +``` + +### Reuse in Config + +```typescript +// playwright.config.ts +use: { + storageState: 'auth.json', +} +``` + +### Setup Project Pattern + +```typescript +// playwright.config.ts +projects: [ + { + name: "setup", + testMatch: /auth.setup\.ts/, + }, + { + name: "chromium", + use: { + ...devices["Desktop Chrome"], + storageState: "playwright/.auth/user.json", + }, + dependencies: ["setup"], + }, +]; +``` + +```typescript +// auth.setup.ts +import { test as setup, expect } from "@playwright/test"; + +const authFile = "playwright/.auth/user.json"; + +setup("authenticate", async ({ page }) => { + await page.goto("/login"); + await page.getByLabel("Email").fill("user@example.com"); + await page.getByLabel("Password").fill("password"); + await page.getByRole("button", { name: "Sign in" }).click(); + await expect(page.getByText("Dashboard")).toBeVisible(); + await page.context().storageState({ path: authFile }); +}); +``` + +### Multiple Roles + +```typescript +// playwright.config.ts +projects: [ + { name: "setup", testMatch: /.*\.setup\.ts/ }, + { + name: "admin", + use: { storageState: "playwright/.auth/admin.json" }, + dependencies: ["setup"], + testMatch: "**/admin/**", + }, + { + name: "user", + use: { storageState: "playwright/.auth/user.json" }, + dependencies: ["setup"], + testMatch: "**/user/**", + }, +]; +``` + +### Per-Test Auth + +```typescript +test.use({ storageState: "admin-auth.json" }); + +test("admin feature", async ({ page }) => { + // Runs with admin auth +}); +``` + +### API Authentication + +```typescript +// auth.setup.ts +setup("authenticate via API", async ({ request }) => { + const response = await request.post("/api/login", { + data: { email: "user@example.com", password: "password" }, + }); + const { token } = await response.json(); + + // Save to storage state + await request.storageState({ path: "auth.json" }); +}); +``` + +### Per-Worker Auth + +```typescript +// fixtures.ts +import { test as base } from "@playwright/test"; + +const users = ["user1@example.com", "user2@example.com", "user3@example.com"]; + +export const test = base.extend({ + account: async ({}, use, testInfo) => { + const email = users[testInfo.parallelIndex % users.length]; + await use({ email, password: "password" }); + }, +}); +``` + +## Fixtures + +### Built-in Fixtures + +- `page` — Isolated page per test +- `context` — Browser context containing page +- `browser` — Shared browser instance +- `browserName` — 'chromium' | 'firefox' | 'webkit' +- `request` — API request context + +### Custom Fixtures + +```typescript +// fixtures.ts +import { test as base, Page } from "@playwright/test"; + +type Fixtures = { + adminPage: Page; + userPage: Page; +}; + +export const test = base.extend({ + adminPage: async ({ browser }, use) => { + const context = await browser.newContext({ + storageState: "admin-auth.json", + }); + const page = await context.newPage(); + await use(page); + await context.close(); + }, + + userPage: async ({ browser }, use) => { + const context = await browser.newContext({ + storageState: "user-auth.json", + }); + const page = await context.newPage(); + await use(page); + await context.close(); + }, +}); + +export { expect } from "@playwright/test"; +``` + +### Fixture with Setup/Teardown + +```typescript +const test = base.extend<{ todoPage: TodoPage }>({ + todoPage: async ({ page }, use) => { + // Setup + const todoPage = new TodoPage(page); + await todoPage.goto(); + await todoPage.createDefaultTodos(); + + // Provide to test + await use(todoPage); + + // Teardown + await todoPage.deleteAllTodos(); + }, +}); +``` + +### Worker-Scoped Fixtures + +Shared across tests in same worker: + +```typescript +const test = base.extend<{}, { dbConnection: Database }>({ + dbConnection: [ + async ({}, use) => { + const db = await Database.connect(); + await use(db); + await db.close(); + }, + { scope: "worker" }, + ], +}); +``` + +### Auto Fixtures + +Run automatically without explicit use: + +```typescript +const test = base.extend({ + logger: [ + async ({}, use) => { + console.log("Test starting"); + await use(); + console.log("Test finished"); + }, + { auto: true }, + ], +}); +``` + +### Fixture Options + +```typescript +const test = base.extend<{ locale: string }>({ + locale: ["en-AU", { option: true }], +}); + +// Override in test +test.use({ locale: "en-US" }); +``` + +## Page Object Model + +### Basic POM + +```typescript +// pages/login.page.ts +import { Page, Locator } from "@playwright/test"; + +export class LoginPage { + readonly page: Page; + readonly emailInput: Locator; + readonly passwordInput: Locator; + readonly submitButton: Locator; + + constructor(page: Page) { + this.page = page; + this.emailInput = page.getByLabel("Email"); + this.passwordInput = page.getByLabel("Password"); + this.submitButton = page.getByRole("button", { name: "Sign in" }); + } + + async goto() { + await this.page.goto("/login"); + } + + async login(email: string, password: string) { + await this.emailInput.fill(email); + await this.passwordInput.fill(password); + await this.submitButton.click(); + } +} +``` + +### Use in Tests + +```typescript +import { test, expect } from "@playwright/test"; +import { LoginPage } from "./pages/login.page"; + +test("user can login", async ({ page }) => { + const loginPage = new LoginPage(page); + await loginPage.goto(); + await loginPage.login("user@example.com", "password"); + await expect(page.getByText("Dashboard")).toBeVisible(); +}); +``` + +### POM as Fixture + +```typescript +// fixtures.ts +import { test as base } from "@playwright/test"; +import { LoginPage } from "./pages/login.page"; +import { DashboardPage } from "./pages/dashboard.page"; + +type Pages = { + loginPage: LoginPage; + dashboardPage: DashboardPage; +}; + +export const test = base.extend({ + loginPage: async ({ page }, use) => { + await use(new LoginPage(page)); + }, + dashboardPage: async ({ page }, use) => { + await use(new DashboardPage(page)); + }, +}); +``` + +```typescript +// login.spec.ts +import { test, expect } from "./fixtures"; + +test("login flow", async ({ loginPage, dashboardPage }) => { + await loginPage.goto(); + await loginPage.login("user@example.com", "password"); + await expect(dashboardPage.welcomeMessage).toBeVisible(); +}); +``` + +## Multiple Pages and Contexts + +### New Page in Same Context + +```typescript +const newPage = await context.newPage(); +await newPage.goto("/other-page"); +``` + +### New Context (Isolated) + +```typescript +const newContext = await browser.newContext(); +const newPage = await newContext.newPage(); +// Has separate cookies, storage +``` + +### Handle Popups + +```typescript +const popupPromise = page.waitForEvent("popup"); +await page.getByRole("link", { name: "Open popup" }).click(); +const popup = await popupPromise; +await popup.waitForLoadState(); +await expect(popup.getByText("Popup content")).toBeVisible(); +``` + +### Multiple Contexts + +```typescript +test("multi-user interaction", async ({ browser }) => { + const userContext = await browser.newContext({ + storageState: "user-auth.json", + }); + const adminContext = await browser.newContext({ + storageState: "admin-auth.json", + }); + + const userPage = await userContext.newPage(); + const adminPage = await adminContext.newPage(); + + // Interact as both users + await userPage.goto("/submit-request"); + await adminPage.goto("/admin/requests"); + + await userContext.close(); + await adminContext.close(); +}); +``` + +## Downloads + +### Wait for Download + +```typescript +const downloadPromise = page.waitForEvent("download"); +await page.getByRole("button", { name: "Download" }).click(); +const download = await downloadPromise; + +// Save to specific path +await download.saveAs("/path/to/save.pdf"); + +// Get suggested filename +const filename = download.suggestedFilename(); + +// Get download path +const path = await download.path(); +``` + +### Configure Downloads + +```typescript +// playwright.config.ts +use: { + acceptDownloads: true, +} +``` + +## Dialogs + +### Handle Alerts + +```typescript +page.on("dialog", (dialog) => dialog.accept()); +await page.getByRole("button", { name: "Delete" }).click(); +``` + +### Handle Confirms + +```typescript +page.on("dialog", (dialog) => { + expect(dialog.message()).toContain("Are you sure?"); + dialog.accept(); +}); +``` + +### Handle Prompts + +```typescript +page.on("dialog", (dialog) => { + dialog.accept("User input"); +}); +``` + +### Dismiss Dialog + +```typescript +page.on("dialog", (dialog) => dialog.dismiss()); +``` + +### One-Time Handler + +```typescript +page.once("dialog", (dialog) => dialog.accept()); +await page.getByRole("button", { name: "Confirm" }).click(); +``` diff --git a/.claude/skills/playwright-testing/references/assertions.md b/.claude/skills/playwright-testing/references/assertions.md new file mode 100644 index 0000000000..f130a95f23 --- /dev/null +++ b/.claude/skills/playwright-testing/references/assertions.md @@ -0,0 +1,334 @@ +# Assertions Reference + +## Contents + +1. [Auto-Retrying Assertions](#auto-retrying-assertions) +2. [Non-Retrying Assertions](#non-retrying-assertions) +3. [Soft Assertions](#soft-assertions) +4. [Polling and Retrying](#polling-and-retrying) +5. [Custom Matchers](#custom-matchers) +6. [Configuration](#configuration) + +## Auto-Retrying Assertions + +These assertions wait and retry until condition is met (default timeout: 5 seconds). + +### Locator State + +```typescript +await expect(locator).toBeVisible(); +await expect(locator).toBeHidden(); +await expect(locator).toBeAttached(); +await expect(locator).toBeEnabled(); +await expect(locator).toBeDisabled(); +await expect(locator).toBeEditable(); +await expect(locator).toBeFocused(); +await expect(locator).toBeChecked(); +await expect(locator).toBeEmpty(); +await expect(locator).toBeInViewport(); +``` + +### Locator Content + +```typescript +await expect(locator).toHaveText("Hello"); +await expect(locator).toHaveText(/hello/i); +await expect(locator).toHaveText(["Item 1", "Item 2"]); // For lists +await expect(locator).toContainText("partial"); +await expect(locator).toHaveValue("input text"); +await expect(locator).toHaveValues(["opt1", "opt2"]); // Multi-select +await expect(locator).toHaveCount(5); +``` + +### Locator Attributes + +```typescript +await expect(locator).toHaveAttribute("href", "/home"); +await expect(locator).toHaveAttribute("href", /^\//); +await expect(locator).toHaveClass("btn-primary"); +await expect(locator).toHaveClass(/active/); +await expect(locator).toContainClass("btn"); +await expect(locator).toHaveId("submit-btn"); +await expect(locator).toHaveCSS("color", "rgb(255, 0, 0)"); +await expect(locator).toHaveJSProperty("checked", true); +``` + +### Accessibility + +```typescript +await expect(locator).toHaveAccessibleName("Submit form"); +await expect(locator).toHaveAccessibleDescription("Click to submit"); +await expect(locator).toHaveRole("button"); +``` + +### Page Assertions + +```typescript +await expect(page).toHaveURL("https://example.com"); +await expect(page).toHaveURL(/dashboard/); +await expect(page).toHaveTitle("Home Page"); +await expect(page).toHaveTitle(/home/i); +``` + +### Visual Comparison + +```typescript +await expect(locator).toHaveScreenshot("button.png"); +await expect(page).toHaveScreenshot("page.png"); +await expect(locator).toMatchAriaSnapshot(` + - button "Submit" + - textbox "Email" +`); +``` + +### Response Assertions + +```typescript +await expect(response).toBeOK(); // Status 200-299 +``` + +### Negation + +```typescript +await expect(locator).not.toBeVisible(); +await expect(locator).not.toHaveText("Error"); +await expect(page).not.toHaveURL(/error/); +``` + +### Custom Timeout + +```typescript +await expect(locator).toBeVisible({ timeout: 10000 }); +``` + +## Non-Retrying Assertions + +Test once without retry—use for static values. + +### Equality + +```typescript +expect(value).toBe(5); // Strict equality (===) +expect(obj).toEqual({ key: "value" }); // Deep equality +expect(obj).toStrictEqual({ key: "v" }); // Deep + type checking +``` + +### Truthiness + +```typescript +expect(value).toBeTruthy(); +expect(value).toBeFalsy(); +expect(value).toBeNull(); +expect(value).toBeUndefined(); +expect(value).toBeDefined(); +expect(value).toBeNaN(); +``` + +### Numbers + +```typescript +expect(num).toBeGreaterThan(5); +expect(num).toBeGreaterThanOrEqual(5); +expect(num).toBeLessThan(10); +expect(num).toBeLessThanOrEqual(10); +expect(0.1 + 0.2).toBeCloseTo(0.3, 5); // Floating point +``` + +### Strings + +```typescript +expect(str).toMatch(/pattern/); +expect(str).toContain("substring"); +``` + +### Collections + +```typescript +expect(arr).toContain("item"); +expect(arr).toContainEqual({ id: 1 }); // Deep equality +expect(arr).toHaveLength(3); +expect(obj).toHaveProperty("key.nested", "value"); +expect(obj).toMatchObject({ partial: "match" }); +``` + +### Functions + +```typescript +expect(() => fn()).toThrow(); +expect(() => fn()).toThrow("error message"); +expect(() => fn()).toThrow(/pattern/); +expect(obj).toBeInstanceOf(MyClass); +``` + +## Soft Assertions + +Continue test execution after failures. Collect all failures before marking test as failed. + +```typescript +await expect.soft(locator).toHaveText("Expected"); +await expect.soft(locator).toBeVisible(); + +// Check if any soft assertions failed +if (test.info().errors.length > 0) { + // Handle accumulated failures +} +``` + +With custom message: + +```typescript +expect.soft(value, "Should be valid ID").toBe(123); +``` + +## Polling and Retrying + +### expect.poll() + +Convert synchronous assertions to polling: + +```typescript +await expect + .poll(async () => { + const response = await page.request.get("/api/status"); + return response.status(); + }) + .toBe(200); + +// With options +await expect + .poll( + async () => { + return await page.evaluate(() => window.loadComplete); + }, + { + intervals: [100, 200, 500], + timeout: 10000, + message: "Should complete loading", + }, + ) + .toBe(true); +``` + +### expect.toPass() + +Retry entire code blocks: + +```typescript +await expect(async () => { + const response = await page.request.get("/api/data"); + expect(response.status()).toBe(200); + const data = await response.json(); + expect(data.items).toHaveLength(10); +}).toPass({ + intervals: [1000, 2000, 5000], + timeout: 30000, +}); +``` + +Note: `toPass` has timeout of 0 by default. + +## Custom Matchers + +### Define Custom Matcher + +```typescript +import { expect as baseExpect } from "@playwright/test"; +import type { Locator } from "@playwright/test"; + +export const expect = baseExpect.extend({ + async toHaveAmount( + locator: Locator, + expected: number, + options?: { timeout?: number }, + ) { + const actual = await locator.getAttribute("data-amount"); + const pass = actual === String(expected); + + return { + pass, + message: () => + pass + ? `Expected amount not to be ${expected}` + : `Expected amount ${expected}, got ${actual}`, + }; + }, +}); +``` + +### Use Custom Matcher + +```typescript +import { test, expect } from "./fixtures"; + +test("check amount", async ({ page }) => { + await expect(page.locator(".cart")).toHaveAmount(4); +}); +``` + +### Merge Multiple Extensions + +```typescript +import { mergeExpects } from "@playwright/test"; +import { expect as dbExpect } from "./db-expects"; +import { expect as a11yExpect } from "./a11y-expects"; + +export const expect = mergeExpects(dbExpect, a11yExpect); +``` + +## Configuration + +### Configure Timeout + +```typescript +// playwright.config.ts +export default defineConfig({ + expect: { + timeout: 10000, // 10 seconds for auto-retrying assertions + }, +}); +``` + +### Pre-configured Expect + +```typescript +// Slow assertions +const slowExpect = expect.configure({ timeout: 30000 }); +await slowExpect(locator).toBeVisible(); + +// Soft by default +const softExpect = expect.configure({ soft: true }); +await softExpect(locator).toHaveText("Value"); +``` + +### Custom Messages + +```typescript +await expect(locator, "Submit button should be visible").toBeVisible(); +expect(value, "User ID should match session").toBe(sessionUserId); +``` + +## Asymmetric Matchers + +Flexible pattern matching: + +```typescript +expect(obj).toEqual({ + id: expect.any(Number), + name: expect.any(String), + email: expect.stringContaining("@"), + tags: expect.arrayContaining(["active"]), + metadata: expect.objectContaining({ version: 1 }), +}); + +expect(value).toEqual(expect.anything()); // Not null/undefined +expect(num).toEqual(expect.closeTo(0.3, 2)); +expect(str).toEqual(expect.stringMatching(/^user_/)); +``` + +## Best Practices + +1. **Always use auto-retrying assertions for web elements** — Prevents flaky tests +2. **Add descriptive messages** — Helps debugging failures +3. **Use soft assertions for comprehensive validation** — Collect multiple failures +4. **Configure appropriate timeouts** — Default 5s may be too short for slow pages +5. **Prefer specific assertions** — `toHaveText` over `toContainText` when possible diff --git a/.claude/skills/playwright-testing/references/configuration.md b/.claude/skills/playwright-testing/references/configuration.md new file mode 100644 index 0000000000..d7f0f0fe4f --- /dev/null +++ b/.claude/skills/playwright-testing/references/configuration.md @@ -0,0 +1,425 @@ +# Configuration Reference + +## Contents + +1. [Basic Configuration](#basic-configuration) +2. [Test Options](#test-options) +3. [Browser Options](#browser-options) +4. [Projects](#projects) +5. [Parallel Execution](#parallel-execution) +6. [Retries](#retries) +7. [Reporters](#reporters) +8. [Web Server](#web-server) +9. [Global Setup/Teardown](#global-setupteardown) +10. [CLI Commands](#cli-commands) + +## Basic Configuration + +```typescript +// playwright.config.ts +import { defineConfig, devices } from "@playwright/test"; + +export default defineConfig({ + testDir: "./tests", + fullyParallel: true, + forbidOnly: !!process.env.CI, + retries: process.env.CI ? 2 : 0, + workers: process.env.CI ? 2 : undefined, + reporter: "html", + use: { + baseURL: "http://localhost:3000", + trace: "on-first-retry", + }, + projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }], +}); +``` + +## Test Options + +### Test Directory and Files + +```typescript +testDir: './tests', // Test files location +testMatch: '**/*test.ts', // Glob pattern for test files +testIgnore: '**/*.skip.ts', // Ignore pattern +``` + +### Timeouts + +```typescript +timeout: 30000, // Per-test timeout (30s default) +globalTimeout: 600000, // Total test run limit +expect: { + timeout: 5000, // Assertion timeout (5s default) + toHaveScreenshot: { timeout: 10000 }, + toMatchSnapshot: { timeout: 10000 }, +}, +``` + +### Output + +```typescript +outputDir: './test-results', // Artifacts folder +preserveOutput: 'failures-only', // 'always' | 'never' | 'failures-only' +snapshotDir: './snapshots', // Screenshot snapshots +snapshotPathTemplate: '{testDir}/__snapshots__/{testFilePath}/{arg}{ext}', +``` + +### Behaviour + +```typescript +forbidOnly: true, // Fail if test.only exists +maxFailures: 10, // Stop after N failures (0 = unlimited) +repeatEach: 1, // Run each test N times +grep: /login/, // Filter tests by title +grepInvert: /skip/, // Exclude tests by title +shard: { current: 1, total: 3 }, // Distribute tests +``` + +## Browser Options + +Configure in `use` section: + +```typescript +use: { + // Browser settings + headless: true, + channel: 'chrome', // 'chrome' | 'msedge' | 'chrome-beta' etc. + launchOptions: { + slowMo: 100, // Slow down actions + args: ['--disable-web-security'], + }, + + // Context settings + viewport: { width: 1280, height: 720 }, + ignoreHTTPSErrors: true, + locale: 'en-AU', + timezoneId: 'Australia/Sydney', + geolocation: { latitude: -33.8688, longitude: 151.2093 }, + permissions: ['geolocation'], + colorScheme: 'dark', // 'light' | 'dark' | 'no-preference' + reducedMotion: 'reduce', + forcedColors: 'active', + + // Network + baseURL: 'http://localhost:3000', + extraHTTPHeaders: { 'X-Custom': 'value' }, + httpCredentials: { username: 'user', password: 'pass' }, + offline: false, + proxy: { server: 'http://proxy:8080' }, + + // Recording + screenshot: 'only-on-failure', // 'off' | 'on' | 'only-on-failure' + video: 'retain-on-failure', // 'off' | 'on' | 'retain-on-failure' + trace: 'on-first-retry', // 'off' | 'on' | 'retain-on-failure' | 'on-first-retry' + + // Timeouts + actionTimeout: 10000, + navigationTimeout: 30000, + + // Test IDs + testIdAttribute: 'data-testid', + + // Storage state + storageState: 'auth.json', +} +``` + +## Projects + +Run tests across different configurations: + +```typescript +projects: [ + // Desktop browsers + { name: "chromium", use: { ...devices["Desktop Chrome"] } }, + { name: "firefox", use: { ...devices["Desktop Firefox"] } }, + { name: "webkit", use: { ...devices["Desktop Safari"] } }, + + // Mobile browsers + { name: "mobile-chrome", use: { ...devices["Pixel 5"] } }, + { name: "mobile-safari", use: { ...devices["iPhone 12"] } }, + + // Branded browsers + { + name: "chrome", + use: { ...devices["Desktop Chrome"], channel: "chrome" }, + }, + { name: "edge", use: { ...devices["Desktop Edge"], channel: "msedge" } }, + + // Custom configuration + { + name: "admin", + use: { + ...devices["Desktop Chrome"], + storageState: "admin-auth.json", + }, + testMatch: "**/admin/*.spec.ts", + }, +]; +``` + +### Project Dependencies + +```typescript +projects: [ + { + name: "setup", + testMatch: /global.setup\.ts/, + }, + { + name: "chromium", + use: { ...devices["Desktop Chrome"] }, + dependencies: ["setup"], + }, +]; +``` + +## Parallel Execution + +### Workers + +```typescript +workers: 4, // Fixed count +workers: '50%', // Percentage of CPU cores +workers: process.env.CI ? 2 : undefined, +``` + +### Parallelisation Modes + +```typescript +fullyParallel: true, // All tests in parallel +``` + +Per-file control: + +```typescript +test.describe.configure({ mode: "parallel" }); +test.describe.configure({ mode: "serial" }); +``` + +### Sharding + +Distribute across machines: + +```bash +npx playwright test --shard=1/3 +npx playwright test --shard=2/3 +npx playwright test --shard=3/3 +``` + +Access worker info: + +```typescript +test("example", async ({ page }, testInfo) => { + console.log(testInfo.workerIndex); // 0, 1, 2... + console.log(testInfo.parallelIndex); // Unique per worker +}); +``` + +## Retries + +```typescript +retries: 2, // Retry failed tests twice +``` + +Per-project: + +```typescript +projects: [ + { + name: "chromium", + retries: 3, + }, +]; +``` + +Per-file: + +```typescript +test.describe.configure({ retries: 2 }); +``` + +Access retry info: + +```typescript +test.beforeEach(async ({ page }, testInfo) => { + if (testInfo.retry > 0) { + // Clean up before retry + } +}); +``` + +## Reporters + +### Built-in Reporters + +```typescript +reporter: 'list', // Line per test (default local) +reporter: 'dot', // Minimal dots (default CI) +reporter: 'line', // Single progress line +reporter: 'html', // Interactive HTML report +reporter: 'json', // JSON output +reporter: 'junit', // JUnit XML +reporter: 'github', // GitHub Actions annotations +reporter: 'blob', // For merging sharded results +``` + +### Multiple Reporters + +```typescript +reporter: [ + ['list'], + ['html', { outputFolder: 'reports/html' }], + ['junit', { outputFile: 'reports/junit.xml' }], +], +``` + +### Reporter Options + +```typescript +reporter: [ + [ + "html", + { + outputFolder: "playwright-report", + open: "never", // 'always' | 'never' | 'on-failure' + }, + ], + [ + "json", + { + outputFile: "test-results.json", + }, + ], + [ + "junit", + { + outputFile: "junit.xml", + embedAnnotationsAsProperties: true, + }, + ], +]; +``` + +## Web Server + +Auto-start dev server before tests: + +```typescript +webServer: { + command: 'npm run dev', + url: 'http://localhost:3000', + reuseExistingServer: !process.env.CI, + timeout: 120000, + stdout: 'pipe', + stderr: 'pipe', +} +``` + +Multiple servers: + +```typescript +webServer: [ + { + command: "npm run dev:frontend", + url: "http://localhost:3000", + }, + { + command: "npm run dev:api", + url: "http://localhost:4000", + }, +]; +``` + +## Global Setup/Teardown + +```typescript +globalSetup: './global-setup.ts', +globalTeardown: './global-teardown.ts', +``` + +Example global setup: + +```typescript +// global-setup.ts +import { chromium, FullConfig } from "@playwright/test"; + +export default async function globalSetup(config: FullConfig) { + const browser = await chromium.launch(); + const page = await browser.newPage(); + + // Perform login + await page.goto("http://localhost:3000/login"); + await page.getByLabel("Email").fill("admin@example.com"); + await page.getByLabel("Password").fill("password"); + await page.getByRole("button", { name: "Login" }).click(); + + // Save auth state + await page.context().storageState({ path: "auth.json" }); + await browser.close(); +} +``` + +## CLI Commands + +### Run Tests + +```bash +npx playwright test # All tests +npx playwright test login.spec.ts # Specific file +npx playwright test tests/auth/ # Directory +npx playwright test -g "login" # Filter by title +npx playwright test --project=chromium # Specific project +npx playwright test --headed # Show browser +npx playwright test --debug # Step-through debugger +npx playwright test --ui # Interactive UI mode +``` + +### Execution Control + +```bash +npx playwright test --workers=4 # Worker count +npx playwright test --retries=2 # Retry count +npx playwright test --max-failures=5 # Stop after failures +npx playwright test --repeat-each=3 # Run each test N times +npx playwright test --shard=1/3 # Sharding +npx playwright test --last-failed # Re-run failures only +``` + +### Output Control + +```bash +npx playwright test --reporter=list +npx playwright test --reporter=json --output=results.json +npx playwright test --trace=on +npx playwright test --screenshot=on +npx playwright test --video=on +``` + +### Update Snapshots + +```bash +npx playwright test --update-snapshots +npx playwright test -u +``` + +### Utilities + +```bash +npx playwright show-report # View HTML report +npx playwright show-trace trace.zip # View trace file +npx playwright codegen # Record tests +npx playwright codegen http://localhost:3000 +npx playwright install # Install browsers +npx playwright install chromium +npx playwright install --with-deps # With system dependencies +``` + +### Environment Variables + +```bash +PWDEBUG=1 npx playwright test # Debug mode +PWDEBUG=console npx playwright test # Console debug helpers +CI=true npx playwright test # CI mode +``` diff --git a/.claude/skills/playwright-testing/references/locators.md b/.claude/skills/playwright-testing/references/locators.md new file mode 100644 index 0000000000..a0ff497138 --- /dev/null +++ b/.claude/skills/playwright-testing/references/locators.md @@ -0,0 +1,268 @@ +# Locators Reference + +## Contents + +1. [Built-in Locators](#built-in-locators) +2. [Chaining and Filtering](#chaining-and-filtering) +3. [List Operations](#list-operations) +4. [Legacy Locators](#legacy-locators) +5. [Framework-Specific Locators](#framework-specific-locators) + +## Built-in Locators + +### Role-Based (Most Recommended) + +```typescript +page.getByRole("button", { name: "Submit" }); +page.getByRole("link", { name: "Home" }); +page.getByRole("textbox", { name: "Email" }); +page.getByRole("checkbox", { name: "Subscribe" }); +page.getByRole("heading", { level: 1 }); +page.getByRole("listitem"); +page.getByRole("dialog"); +page.getByRole("tab", { selected: true }); +``` + +Options: + +- `name` — Accessible name (exact or regex) +- `exact` — Whether name match is exact (default: false) +- `checked` — For checkboxes/radios +- `disabled` — Match disabled state +- `expanded` — For expandable elements +- `level` — For headings (1-6) +- `pressed` — For toggle buttons +- `selected` — For options/tabs + +### Form Labels + +```typescript +page.getByLabel("Email address"); +page.getByLabel("Password", { exact: true }); +page.getByLabel(/remember me/i); +``` + +Associates form controls via `