Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fd5703f
refactor(test): ♻️ migrate acc_helpers, helpers test via rename
tmeckel Jan 5, 2026
2004c22
refactor(util): ♻️ migrate test/poll -> util/poll and tests
tmeckel Jan 5, 2026
3672938
refactor(util): ♻️ introduce string builder helper
tmeckel Jan 5, 2026
5ec1945
feat(serviceendpoint-shared): ✨ introduce common create flags/options
tmeckel Jan 5, 2026
e663d8d
test(serviceendpoint-shared): ✅ add tests for common create flags
tmeckel Jan 5, 2026
a48f8f8
feat(serviceendpoint-shared): ✨ add shared update flags/options
tmeckel Jan 5, 2026
a43c266
test(serviceendpoint-shared): ✅ add tests for common update flags
tmeckel Jan 5, 2026
ab14f0a
feat(serviceendpoint-shared): ✨ add shared create runner
tmeckel Jan 5, 2026
24b616c
feat(serviceendpoint-shared): ✨ add shared update runner
tmeckel Jan 5, 2026
be6f733
feat(serviceendpoint-shared): ✨ add type registry for endpoint handlers
tmeckel Jan 5, 2026
e6a5b9d
feat(serviceendpoint-shared): ✨ add endpoint type metadata validator
tmeckel Jan 5, 2026
3b1fb03
feat(serviceendpoint-shared): ✨ add shared test connection helper
tmeckel Jan 5, 2026
6ed0369
test(serviceendpoint-shared): ✅ add tests for test connection logic
tmeckel Jan 5, 2026
b363ea1
feat(serviceendpoint-shared): ✨ add wait-for-ready helper
tmeckel Jan 5, 2026
363cdc5
test(serviceendpoint-shared): ✅ add tests for wait-ready helper
tmeckel Jan 5, 2026
e74543a
docs(serviceendpoint): 🗒️ add developer README for service endpoints
tmeckel Jan 5, 2026
352288c
refactor(cmd): ♻️ refactor GitHub create to shared configurer pattern
tmeckel Jan 5, 2026
88e7f4f
test(cmd): ✅ update GitHub create acceptance tests for shared runner
tmeckel Jan 5, 2026
8f49bf1
test(cmd): ✅ update GitHub create unit tests for configurer
tmeckel Jan 5, 2026
44ff753
refactor(cmd): ♻️ refactor AzureRM create to shared configurer pattern
tmeckel Jan 5, 2026
ac79abe
test(cmd): ✅ update AzureRM create acceptance tests for shared runner
tmeckel Jan 5, 2026
b6a920c
test(security-permission): ✅ switch delete acceptance tests to new po…
tmeckel Jan 5, 2026
68389dc
test(security-permission): ✅ switch reset acceptance tests to new pol…
tmeckel Jan 5, 2026
b9739f4
test(security-permission): ✅ switch update acceptance tests to new po…
tmeckel Jan 5, 2026
f383adb
test(serviceendpoint-delete): ✅ align delete acceptance tests with re…
tmeckel Jan 5, 2026
3eef240
docs: 📄 add new flags for connection validation and timeout
tmeckel Jan 5, 2026
f9bf549
docs: 📄 update acceptance testing guidelines
tmeckel Jan 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 33 additions & 5 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,15 @@ Use this pattern as a seed and add more mocks only as the code under test demand

Acceptance tests (`*_acc_test.go`) run against a live Azure DevOps organization using the harness under `internal/test`. They are opt-in and should only be used when unit tests and mocks cannot provide enough confidence.

### Getting started (`AcceptanceTest`)

`inttest.Test` executes a `TestCase` using a `TestContext` created by the harness. The `TestCase.AcceptanceTest` boolean controls which context is used:

- `AcceptanceTest: true` (recommended for all `*_acc_test.go`): uses `inttest.NewAccTestContext(t)` which reads the required `AZDO_ACC_*` environment variables and creates real SDK clients for the configured organization.
- `AcceptanceTest: false`: uses `inttest.NewTestContext(t)` which generates random placeholder values for `Org`, `PAT`, and `Project`. This is only useful for hermetic unit tests that want a lightweight `util.CmdContext` implementation; it is not suitable for live Azure DevOps operations.

For acceptance tests, always set `AcceptanceTest: true`. Otherwise the harness will still be gated by `AZDO_ACC_TEST=1`, but the generated placeholder org/token values will typically cause confusing failures once your steps attempt real API calls.

### When to add an acceptance test

- You need to verify a workflow/command (e.g., modifying security permissions) against real data.
Expand All @@ -92,20 +101,39 @@ Acceptance tests (`*_acc_test.go`) run against a live Azure DevOps organization
| ------------------ | ------------------------------------------------------------------------------------------------------ |
| `AZDO_ACC_TEST=1` | Enables acceptance tests. Without it, `inttest.Test` skips all steps. |
| `AZDO_ACC_ORG` | Organization name used for the session. |
| `AZDO_ACC_ORG_URL` | Optional explicit organization URL; defaults to `https://dev.azure.com/<org>`. |
| `AZDO_ACC_PAT` | Personal Access Token with the scopes required by the test steps. |
| `AZDO_ACC_PROJECT` | Project name used by acceptance tests that operate on project-scoped resources. |
| `AZDO_ACC_TIMEOUT` | Optional override for the default 60 s timeout. Accepts Go durations (`45s`, `2m`) or integer seconds. |
| `AZDO_ACC_TIMEOUT` | Optional override for the default 240 s timeout. Accepts Go durations (`45s`, `2m`) or integer seconds. Use `-1` to disable timeouts. |

### Step-by-step skeleton

1. Place the test in the command package with the `_acc_test.go` suffix.
2. Wrap your steps in `inttest.Test(t, inttest.TestCase{ Steps: []inttest.Step{ ... } })`.
2. Wrap your steps in `inttest.Test(t, inttest.TestCase{ AcceptanceTest: true, Steps: []inttest.Step{ ... } })`.
3. **PreRun**: create or seed live resources (groups, repositories, permissions) using `ctx.ClientFactory()`.
4. **Run**: construct the command options and call the command’s `run...` helper directly (e.g., `return runCommand(ctx, opts)`).
5. **Verify**: use `inttest.Poll` to wait for eventual consistency and assert desired state.
5. **Verify**: use `internal/util/poll.go` (`pollutil.Poll(ctx.Context(), ...)`) to wait for eventual consistency and assert desired state.
6. **PostRun**: delete or revert all resources you created; aggregate cleanup errors with `errors.Join`.

Minimal scaffold:

```go
inttest.Test(t, inttest.TestCase{
AcceptanceTest: true,
PreCheck: func() error {
// Validate required inputs (e.g., ensure AZDO_ACC_PROJECT is set when needed).
return nil
},
Steps: []inttest.Step{
{
PreRun: func(ctx inttest.TestContext) error { return nil },
Run: func(ctx inttest.TestContext) error { return nil },
Verify: func(ctx inttest.TestContext) error { return nil },
PostRun: func(ctx inttest.TestContext) error { return nil },
},
},
})
```

Example shell to execute a single acceptance test:
```bash
AZDO_ACC_TEST=1 \
Expand All @@ -119,7 +147,7 @@ Acceptance tests are not run in CI; execute them manually before publishing feat

- `inttest.TestContext` now exposes `Project()` alongside `Org`, `OrgUrl`, and `PAT`. Set `AZDO_ACC_PROJECT` when a test needs to target a specific project and fail fast in `PreRun` if it is missing.
- Use `TestContext.SetValue(key, value)`/`Value(key)` to propagate data across `PreRun`, `Run`, `Verify`, and `PostRun` without relying on package-level variables. Keys can be simple strings or typed aliases; mimic `context.Context` usage.
- The helper `inttest.WriteTestFile(path, contents)` creates or truncates files with `0600` permissions and ensures parent directories exist, which is useful for acceptance tests that need temporary credentials or certificates.
- The helpers `inttest.WriteTestFile(t, contents)` and `inttest.WriteTestFileWithName(t, filename, contents)` create files with `0600` permissions under `t.TempDir()`, which is useful for acceptance tests that need temporary credentials or certificates.

### Updating Mocks

Expand Down
25 changes: 17 additions & 8 deletions docs/azdo_help_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -1089,7 +1089,10 @@ Create an Azure Resource Manager service connection
--subscription-name string Azure subscription name
-t, --template string Format JSON output using a Go template; see "azdo help formatting"
--tenant-id string Azure tenant ID (e.g., GUID)
-y, --yes Skip confirmation prompts
--timeout duration Maximum time to wait when --wait or --validate-connection is enabled (default 2m0s)
--validate-connection Run TestConnection after creation (opt-in)
--validate-schema Validate auth scheme/params against endpoint type metadata (opt-in)
--wait Wait until the endpoint reports ready/failed
```

Aliases
Expand All @@ -1103,13 +1106,19 @@ cr, c, new, n, add, a
Create a GitHub service endpoint

```
--configuration-id string Configuration for connecting to the endpoint (use an OAuth/installation configuration). Mutually exclusive with --token.
-q, --jq expression Filter JSON output using a jq expression
--json fields[=*] Output JSON with the specified fields. Prefix a field with '-' to exclude it.
--name string Name of the service endpoint
-t, --template string Format JSON output using a Go template; see "azdo help formatting"
--token string Visit https://github.com/settings/tokens to create personal access tokens. Recommended scopes: repo, user, admin:repo_hook. If omitted, you will be prompted for a token when interactive.
--url string GitHub URL (defaults to https://github.com)
--configuration-id string Configuration for connecting to the endpoint (use an OAuth/installation configuration). Mutually exclusive with --token.
--description string Description for the service endpoint
--grant-permission-to-all-pipelines Grant access permission to all pipelines to use the service connection
-q, --jq expression Filter JSON output using a jq expression
--json fields[=*] Output JSON with the specified fields. Prefix a field with '-' to exclude it.
--name string Name of the service endpoint
-t, --template string Format JSON output using a Go template; see "azdo help formatting"
--timeout duration Maximum time to wait when --wait or --validate-connection is enabled (default 2m0s)
--token string Visit https://github.com/settings/tokens to create personal access tokens. Recommended scopes: repo, user, admin:repo_hook. If omitted, you will be prompted for a token when interactive.
--url string GitHub URL (defaults to https://github.com)
--validate-connection Run TestConnection after creation (opt-in)
--validate-schema Validate auth scheme/params against endpoint type metadata (opt-in)
--wait Wait until the endpoint reports ready/failed
```

### `azdo service-endpoint delete [ORGANIZATION/]PROJECT/ID_OR_NAME [flags]`
Expand Down
16 changes: 14 additions & 2 deletions docs/azdo_service-endpoint_create_azurerm.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,21 @@ This command is modeled after the Azure DevOps Terraform Provider's implementati

Azure tenant ID (e.g., GUID)

* `-y`, `--yes`
* `--timeout` `duration` (default `&#34;2m0s&#34;`)

Skip confirmation prompts
Maximum time to wait when --wait or --validate-connection is enabled

* `--validate-connection`

Run TestConnection after creation (opt-in)

* `--validate-schema`

Validate auth scheme/params against endpoint type metadata (opt-in)

* `--wait`

Wait until the endpoint reports ready/failed


### ALIASES
Expand Down
24 changes: 24 additions & 0 deletions docs/azdo_service-endpoint_create_github.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@ Create a GitHub service endpoint using a personal access token (PAT) or an insta

Configuration for connecting to the endpoint (use an OAuth/installation configuration). Mutually exclusive with --token.

* `--description` `string`

Description for the service endpoint

* `--grant-permission-to-all-pipelines`

Grant access permission to all pipelines to use the service connection

* `-q`, `--jq` `expression`

Filter JSON output using a jq expression
Expand All @@ -30,6 +38,10 @@ Create a GitHub service endpoint using a personal access token (PAT) or an insta

Format JSON output using a Go template; see &#34;azdo help formatting&#34;

* `--timeout` `duration` (default `&#34;2m0s&#34;`)

Maximum time to wait when --wait or --validate-connection is enabled

* `--token` `string`

Visit https://github.com/settings/tokens to create personal access tokens. Recommended scopes: repo, user, admin:repo_hook. If omitted, you will be prompted for a token when interactive.
Expand All @@ -38,6 +50,18 @@ Create a GitHub service endpoint using a personal access token (PAT) or an insta

GitHub URL (defaults to https://github.com)

* `--validate-connection`

Run TestConnection after creation (opt-in)

* `--validate-schema`

Validate auth scheme/params against endpoint type metadata (opt-in)

* `--wait`

Wait until the endpoint reports ready/failed


### JSON Fields

Expand Down
6 changes: 4 additions & 2 deletions internal/cmd/security/permission/delete/delete_acc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"github.com/tmeckel/azdo-cli/internal/cmd/security/permission/shared"
inttest "github.com/tmeckel/azdo-cli/internal/test"
"github.com/tmeckel/azdo-cli/internal/types"
pollutil "github.com/tmeckel/azdo-cli/internal/util"
)

type aceContainer struct {
Expand All @@ -31,6 +32,7 @@ func TestAccDeletePermission(t *testing.T) {
var groupIdentity string

inttest.Test(t, inttest.TestCase{
AcceptanceTest: true,
Steps: []inttest.Step{
{
PreRun: func(ctx inttest.TestContext) error {
Expand Down Expand Up @@ -103,7 +105,7 @@ func TestAccDeletePermission(t *testing.T) {
if err != nil {
return err
}
return inttest.Poll(func() error {
return pollutil.Poll(ctx.Context(), func() error {
ace, err := shared.FindAccessControlEntry(ctx.Context(), secClient, nsUUID, token, groupIdentity)
if err != nil {
return err
Expand All @@ -117,7 +119,7 @@ func TestAccDeletePermission(t *testing.T) {
return nil
}
return fmt.Errorf("expected ACE to be removed; allow=%d deny=%d", allow, deny)
}, inttest.PollOptions{
}, pollutil.PollOptions{
Tries: 10,
Timeout: 30 * time.Second,
})
Expand Down
6 changes: 4 additions & 2 deletions internal/cmd/security/permission/reset/reset_acc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"github.com/tmeckel/azdo-cli/internal/cmd/security/permission/shared"
inttest "github.com/tmeckel/azdo-cli/internal/test"
"github.com/tmeckel/azdo-cli/internal/types"
pollutil "github.com/tmeckel/azdo-cli/internal/util"
)

type aceContainer struct {
Expand Down Expand Up @@ -44,6 +45,7 @@ func TestAccResetPermission(t *testing.T) {
)

inttest.Test(t, inttest.TestCase{
AcceptanceTest: true,
Steps: []inttest.Step{
{
PreRun: func(ctx inttest.TestContext) error {
Expand Down Expand Up @@ -119,7 +121,7 @@ func TestAccResetPermission(t *testing.T) {
return err
}

return inttest.Poll(func() error {
return pollutil.Poll(ctx.Context(), func() error {
ace, err := shared.FindAccessControlEntry(ctx.Context(), secClient, nsUUID, token, groupIdentity)
if err != nil {
return err
Expand All @@ -135,7 +137,7 @@ func TestAccResetPermission(t *testing.T) {
return fmt.Errorf("unexpected permission state allow=0x%X deny=0x%X", allow, deny)
}
return nil
}, inttest.PollOptions{
}, pollutil.PollOptions{
Tries: 10,
Timeout: 30 * time.Second,
})
Expand Down
6 changes: 4 additions & 2 deletions internal/cmd/security/permission/update/update_acc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (

"github.com/tmeckel/azdo-cli/internal/cmd/security/permission/shared"
inttest "github.com/tmeckel/azdo-cli/internal/test"
pollutil "github.com/tmeckel/azdo-cli/internal/util"
)

func TestAccUpdatePermission(t *testing.T) {
Expand All @@ -26,6 +27,7 @@ func TestAccUpdatePermission(t *testing.T) {
groupName := fmt.Sprintf("azdo-cli-test-group-%s", uuid.New().String())

inttest.Test(t, inttest.TestCase{
AcceptanceTest: true,
Steps: []inttest.Step{
{
PreRun: func(ctx inttest.TestContext) error {
Expand Down Expand Up @@ -72,7 +74,7 @@ func TestAccUpdatePermission(t *testing.T) {
if err != nil {
return err
}
return inttest.Poll(func() error {
return pollutil.Poll(ctx.Context(), func() error {
ace, err := shared.FindAccessControlEntry(ctx.Context(), sec, nsUUID, token, groupIdentity)
if err != nil {
return err
Expand All @@ -87,7 +89,7 @@ func TestAccUpdatePermission(t *testing.T) {
return fmt.Errorf("allow mask %d does not contain expected bit 0x2", *ace.Allow)
}
return nil
}, inttest.PollOptions{
}, pollutil.PollOptions{
Tries: 10,
Timeout: 30 * time.Second,
})
Expand Down
Loading
Loading