Merged
Conversation
Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>
…ET 10 Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>
Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>
Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>
… but the MassTransit.ActiveMQ package isn't updated yet Tracked: MassTransit/MassTransit#6133
…ss ambiguous referencing
I feel dirty doing that...
aaronpowell
added
the
awaiting response
Youssef1313
reviewed
Nov 16, 2025
.github/workflows/tests.yaml
Outdated
| - name: Run tests | ||
| run: >- | ||
| dotnet test ${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }} | ||
| dotnet run ${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }} |
There was a problem hiding this comment.
You need to:
- Remove
--logger "console;verbosity=normal". - Replace
--logger trxwith--report-trx(and ensure Microsoft.Testing.Extensions.TrxReport is referenced) - Replace all the blame-related args with
--crashdump,--hangdump, and--hangdump-timeout 7m --results-directoryis changing the meaning a little bit with this PR. Previously it would be relative to current working directory${{ github.workspace }}while with this PR it will be relative to the test executable. So either pass the full path, or keep usingdotnet testwhich will handle the path transformation for you.--collect "XPlat Code Coverage"needs to be replaced with--coverage(and ensure Microsoft.Testing.Extensions.CodeCoverage is referenced)
Youssef1313
reviewed
Nov 16, 2025
.github/workflows/tests.yaml
Outdated
| --report-trx | ||
| --report-trx-filename "${{ matrix.name }}-${{ matrix.os }}.trx" | ||
| --ignore-exit-code 8 | ||
| -- RunConfiguration.CollectSourceInformation=true |
Youssef1313
reviewed
Nov 16, 2025
Directory.Packages.props
Outdated
| <PackageVersion Include="xunit.runner.visualstudio" Version="2.8.2" /> | ||
| <PackageVersion Include="xunit.extensibility.execution" Version="2.9.3" /> | ||
| <PackageVersion Include="Microsoft.DotNet.XUnitExtensions" Version="9.0.0-beta.24568.1" /> | ||
| <PackageVersion Include="xunit.v3" Version="3.2.0" /> |
There was a problem hiding this comment.
nit: there is now xunit.v3.mtp-v2 package.
Youssef1313
reviewed
Nov 16, 2025
| <PackageVersion Include="xunit.v3" Version="3.2.0" /> | ||
| <PackageVersion Include="xunit.v3.assert" Version="3.2.0" /> | ||
| <PackageVersion Include="xunit.v3.extensibility.core" Version="3.2.0" /> | ||
| <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" /> |
There was a problem hiding this comment.
nit: no longer needed if you don't care about VSTest support.
There was a problem hiding this comment.
Suggested change
| <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" /> |
Youssef1313
reviewed
Nov 16, 2025
tests/.runsettings
Outdated
| <RunConfiguration> | ||
| <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests --> | ||
| <TestCaseFilter>category!=unsupported-platform</TestCaseFilter> | ||
| <TestCaseFilter>category!=failing</TestCaseFilter> |
There was a problem hiding this comment.
runsettings isn't supported with MTP.
Youssef1313
reviewed
Nov 16, 2025
tests/Directory.Build.props
Outdated
| https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes --> | ||
| <TestRunnerAdditionalArguments>$(TestRunnerAdditionalArguments) --ignore-exit-code 8</TestRunnerAdditionalArguments> | ||
|
|
||
| <!-- <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner> --> |
Youssef1313
reviewed
Nov 16, 2025
Youssef1313
left a comment
Youssef1313
left a comment
There was a problem hiding this comment.
You need to also update global.json to specify the test runner as MTP, and probably ensure it requires .NET 10.
aaronpowell
temporarily deployed
to
azure-artifacts
GitHub Actions
Inactive
The xUnit v3 MTP runner's --coverage flag defaults to binary .coverage format. Add --coverage-output-format cobertura and --coverage-output to generate .cobertura.xml files that the ReportGenerator step expects. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
aaronpowell
temporarily deployed
to
azure-artifacts
GitHub Actions
Inactive
aaronpowell
requested review from
Alirexaa,
Harold-Morgan,
aviv86,
justinyoo,
karmeli87,
martinjt,
ppekrol and
shiranshalom
as code owners
aaronpowell
requested review from
ErikEJ,
FullStackChef,
MichielBrys,
WhitWaldo,
esskar,
fboucher,
jmezach,
paule96 and
tommasodotNET
as code owners
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Upgrades the repository’s test infrastructure from xUnit v2 to xUnit v3, aligning with Aspire’s current testing stack and moving execution to the Microsoft Testing Platform (MTP), including CI/workflow and local tooling adjustments.
Changes:
- Migrate test projects to xUnit v3 (including updated trait infrastructure and async lifetime signatures).
- Switch test execution to Microsoft Testing Platform (packages + runner configuration) and update CI to use MTP reporting/coverage options.
- Add supporting tooling/config changes (clean script, VS Code task, ignores) and minor PowerShell hosting adjustments to avoid test hangs.
Reviewed changes
Copilot reviewed 62 out of 63 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Directory.Build.props | Switch test dependencies to xUnit v3 + MTP packages; configure runner args and filtering. |
| tests/CommunityToolkit.Aspire.Testing/TestDistributedApplicationBuilder.cs | Remove now-unneeded xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Testing/RequiresWindowsDiscoverer.cs | Remove xUnit v2 trait discoverer (v3 trait approach). |
| tests/CommunityToolkit.Aspire.Testing/RequiresWindowsAttribute.cs | Rework trait handling for xUnit v3 via ITraitAttribute.GetTraits(). |
| tests/CommunityToolkit.Aspire.Testing/RequiresLinuxDiscoverer.cs | Remove xUnit v2 trait discoverer (v3 trait approach). |
| tests/CommunityToolkit.Aspire.Testing/RequiresLinuxAttribute.cs | Rework trait handling for xUnit v3 via ITraitAttribute.GetTraits(). |
| tests/CommunityToolkit.Aspire.Testing/RequiresDockerDiscoverer.cs | Remove xUnit v2 trait discoverer (v3 trait approach). |
| tests/CommunityToolkit.Aspire.Testing/RequiresDockerAttribute.cs | Rework trait handling for xUnit v3; adjust namespace/usage. |
| tests/CommunityToolkit.Aspire.Testing/RequiresAuthenticatedToolDiscoverer.cs | Remove xUnit v2 trait discoverer (v3 trait approach). |
| tests/CommunityToolkit.Aspire.Testing/RequiresAuthenticatedToolAttribute.cs | Rework trait handling for xUnit v3 via ITraitAttribute.GetTraits(). |
| tests/CommunityToolkit.Aspire.Testing/PlatformDetection.cs | Expand CI/platform detection helpers used by test traits. |
| tests/CommunityToolkit.Aspire.Testing/DistributedApplicationTestingBuilderExtensions.cs | Remove now-unneeded xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Testing/ConformanceTests.cs | Replace Conditional* attributes and SkipTestException patterns with xUnit v3 equivalents. |
| tests/CommunityToolkit.Aspire.Testing/CommunityToolkit.Aspire.Testing.csproj | Update xUnit extension packages for v3 and MTP compatibility. |
| tests/CommunityToolkit.Aspire.Testing/AspireIntegrationTest.cs | Update async lifetime signature to xUnit v3 ValueTask. |
| tests/CommunityToolkit.Aspire.SurrealDb.Tests/SurrealDbContainerFixture.cs | Update fixture async lifetime signatures to ValueTask; add testing utilities import. |
| tests/CommunityToolkit.Aspire.SurrealDb.Tests/ConformanceTests.cs | Adjust conformance hooks for xUnit v3 + updated test utilities. |
| tests/CommunityToolkit.Aspire.SurrealDb.Tests/AspireSurrealClientExtensionsTest.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.RavenDB.Client.Tests/RavenDbServerFixture.cs | Update fixture async lifetime signatures to ValueTask. |
| tests/CommunityToolkit.Aspire.RavenDB.Client.Tests/CommunityToolkit.Aspire.RavenDB.Client.Tests.csproj | Remove redundant per-project nullable/implicit-using settings (inherited globally). |
| tests/CommunityToolkit.Aspire.OllamaSharp.Tests/OllamaContainerFeature.cs | Update fixture async lifetime signatures to ValueTask; add testing utilities import. |
| tests/CommunityToolkit.Aspire.OllamaSharp.Tests/ConformanceTests.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.Minio.Client.Tests/MinioContainerFeature.cs | Update fixture async lifetime signatures to ValueTask; add testing utilities import. |
| tests/CommunityToolkit.Aspire.Minio.Client.Tests/ConformanceTests.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.Meilisearch.Tests/MeilisearchContainerFixture.cs | Update fixture async lifetime signatures to ValueTask; add testing utilities import. |
| tests/CommunityToolkit.Aspire.Meilisearch.Tests/ConformanceTests.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.Meilisearch.Tests/AspireMeilisearchClientExtensionsTest.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.KurrentDB.Tests/KurrentDBContainerFixture.cs | Update fixture async lifetime signatures to ValueTask; add testing utilities import. |
| tests/CommunityToolkit.Aspire.KurrentDB.Tests/ConformanceTests.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.KurrentDB.Tests/AspireKurrentDBClientExtensionsTest.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.Hosting.SurrealDb.Tests/SurrealDbFunctionalTests.cs | Remove now-unneeded xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.Stripe.Tests/AppHostTests.cs | Align imports with new shared testing utilities. |
| tests/CommunityToolkit.Aspire.Hosting.SqlServer.Extensions.Tests/ResourceCreationTests.cs | Update expected annotation keys (likely due to runner/resource naming changes). |
| tests/CommunityToolkit.Aspire.Hosting.SqlDatabaseProjects.Tests/SqlServerContainerFixture.cs | Update fixture async lifetime signatures to ValueTask. |
| tests/CommunityToolkit.Aspire.Hosting.SqlDatabaseProjects.Tests/FunctionalTests.cs | Align imports with shared testing utilities; remove xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.Sftp.Tests/SftpFunctionalTests.cs | Align imports with shared testing utilities; remove xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.Sftp.Tests/AppHostTests.cs | Remove now-unneeded xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.Redis.Extensions.Tests/ResourceCreationTests.cs | Update expected annotation keys (likely due to runner/resource naming changes). |
| tests/CommunityToolkit.Aspire.Hosting.PowerShell.Tests/AppHostTests.cs | Minor formatting/import adjustments for test suite changes. |
| tests/CommunityToolkit.Aspire.Hosting.OpenTelemetryCollector.Tests/ResourceCreationTests.cs | Swap xUnit abstractions import for shared testing utilities import. |
| tests/CommunityToolkit.Aspire.Hosting.MongoDB.Extensions.Tests/ResourceCreationTests.cs | Remove manual endpoint allocation calls from tests. |
| tests/CommunityToolkit.Aspire.Hosting.Minio.Tests/MinioFunctionalTests.cs | Align imports with shared testing utilities; remove xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.Meilisearch.Tests/MeilisearchFunctionalTests.cs | Align imports with shared testing utilities; remove xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.McpInspector.Tests/AppHostTests.cs | Remove Linux-only test marker comment/attribute. |
| tests/CommunityToolkit.Aspire.Hosting.KurrentDB.Tests/KurrentDBFunctionalTests.cs | Align imports with shared testing utilities; remove xUnit abstractions import. |
| tests/CommunityToolkit.Aspire.Hosting.GoFeatureFlag.Tests/GoFeatureFlagFunctionalTests.cs | Align imports with shared testing utilities; remove legacy imports. |
| tests/CommunityToolkit.Aspire.Hosting.DbGate.Tests/AddDbGateTests.cs | Remove manual endpoint allocation calls from tests. |
| tests/CommunityToolkit.Aspire.GoFeatureFlag.Tests/GoFeatureFlagContainerFixture.cs | Update fixture async lifetime signatures to ValueTask; add testing utilities import. |
| tests/CommunityToolkit.Aspire.GoFeatureFlag.Tests/ConformanceTests.cs | Add testing utilities import for updated traits/fixtures. |
| tests/CommunityToolkit.Aspire.GoFeatureFlag.Tests/AspireGoFeatureFlagClientExtensionsTest.cs | Add testing utilities import for updated traits/fixtures. |
| tests/.runsettings | Remove VSTest runsettings filtering (replaced by MTP filtering). |
| src/CommunityToolkit.Aspire.Hosting.PowerShell/PowerShellRunspacePoolResourceBuilderExtensions.cs | Refactor initialization hook usage to newer builder APIs. |
| src/CommunityToolkit.Aspire.Hosting.PowerShell/PowerShellRunspacePoolResource.cs | Close runspace pool on app stop to prevent xUnit v3 hangs. |
| src/CommunityToolkit.Aspire.Hosting.PowerShell/DistributedApplicationBuilderExtensions.cs | Refactor initialization, state setup, and inject host lifetime for cleanup. |
| nuget.config | Update package source mapping for xUnit v3 extension package. |
| examples/powershell/CommunityToolkit.Aspire.PowerShell.AppHost/Properties/launchSettings.json | Add diagnostics log level environment variable. |
| examples/powershell/CommunityToolkit.Aspire.PowerShell.AppHost/Program.cs | Adjust Azurite args and script path/cwd handling for example reliability. |
| eng/clean-bin-obj.sh | Add repo cleanup helper script for removing bin/obj directories. |
| Directory.Packages.props | Update package versions/references for xUnit v3 + MTP + logger/reporting extensions. |
| .vscode/tasks.json | Reformat tasks and add cleanup task invoking the new script. |
| .gitignore | Normalize ignoring of test results folder casing. |
| .github/workflows/tests.yaml | Update CI to run tests via dotnet run using MTP reporting/coverage options. |
| .github/actions/setup-runtimes-caching/action.yml | Minor formatting (trailing newline). |
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+6
to
+9
| /// <summary> | ||
| /// Marks a test or test class as requiring a Linux operating system. | ||
| /// Adds a trait so tests can be skipped or filtered when not running on Linux. | ||
| /// </summary> |
There was a problem hiding this comment.
The XML summary is incorrect: this is RequiresWindowsAttribute but the comment says the test requires a Linux OS. Update the summary text to refer to Windows (and adjust the “not running on Linux” phrasing accordingly) to avoid misleading consumers.
Comment on lines
268
to
272
| public void LoggerFactoryIsUsedByRegisteredClient(bool registerAfterLoggerFactory, bool useKey) | ||
| { | ||
| SkipIfRequiredServerConnectionCanNotBeEstablished(); | ||
| SkipIfKeyedRegistrationIsNotSupported(useKey); | ||
|
|
There was a problem hiding this comment.
LoggerFactoryIsUsedByRegisteredClient no longer skips when both RequiredLogCategories and NotAcceptableLogCategories are empty, which makes the assertions a no-op and the test always pass without validating anything. Consider restoring the skip (e.g., via Assert.Skip) when there are no categories to check.
Comment on lines
107
to
112
| protected override void TriggerActivity(SurrealDbClient service) | ||
| { | ||
| using var source = new CancellationTokenSource(100); | ||
|
|
||
| service.Version(source.Token).Wait(); | ||
| Task.Run(() => service.Connect()); | ||
| } |
There was a problem hiding this comment.
TriggerActivity now starts service.Connect() via Task.Run(...) and immediately returns, and the created CancellationTokenSource is unused. This makes the test non-deterministic (and may not trigger any activity before assertions). Prefer a deterministic call (await the operation or block on the returned task) and either use the token or remove the CTS.
Comment on lines
+107
to
+121
| dotnet run | ||
| --project "${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }}" | ||
| --no-build | ||
| --configuration ${{ env.DOTNET_CONFIGURATION }} | ||
| --logger "console;verbosity=normal" | ||
| --logger "trx" | ||
| --logger "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true" | ||
| --blame | ||
| --blame-hang-timeout 7m | ||
| --blame-crash | ||
| --results-directory testresults | ||
| --collect "XPlat Code Coverage" | ||
| --no-restore | ||
| --no-build -- RunConfiguration.CollectSourceInformation=true | ||
| --crashdump | ||
| --hangdump | ||
| --hangdump-timeout 7m | ||
| --results-directory "${{ github.workspace }}/testresults" | ||
| --filter-not-trait "category=failing" | ||
| --report-trx | ||
| --report-trx-filename "${{ matrix.name }}-${{ matrix.os }}.trx" | ||
| --coverage | ||
| --coverage-output-format cobertura | ||
| --coverage-output coverage.cobertura.xml | ||
| --report-github |
There was a problem hiding this comment.
The dotnet run invocation is missing the -- separator before application/test runner arguments. As written, options like --crashdump/--hangdump will be parsed as dotnet run options and the command will fail. Add -- after the dotnet run options (e.g., after --configuration ...) so the remaining arguments are passed to the test executable.
Comment on lines
+55
to
+57
| // if we don't do this, xunit 3 will hang on exit because of open runspaces (foreground threads) | ||
| logger.LogInformation("Closing PowerShell runspace pool '{PoolName}'", Name); | ||
| Pool?.Close(); |
There was a problem hiding this comment.
The ApplicationStopping.Register callback calls Pool?.Close() without any exception handling. If Close() throws (e.g., pool already disposed/broken), it can surface during shutdown and potentially interfere with host stopping. Wrap the callback body in a try/catch (and consider disposing the pool) to make shutdown more robust.
Suggested change
| // if we don't do this, xunit 3 will hang on exit because of open runspaces (foreground threads) | |
| logger.LogInformation("Closing PowerShell runspace pool '{PoolName}'", Name); | |
| Pool?.Close(); | |
| try | |
| { | |
| // if we don't do this, xunit 3 will hang on exit because of open runspaces (foreground threads) | |
| logger.LogInformation("Closing PowerShell runspace pool '{PoolName}'", Name); | |
| if (Pool is not null) | |
| { | |
| Pool.Close(); | |
| Pool.Dispose(); | |
| Pool = null; | |
| } | |
| } | |
| catch (ObjectDisposedException ex) | |
| { | |
| logger.LogDebug(ex, "PowerShell runspace pool '{PoolName}' was already disposed during application stopping.", Name); | |
| } | |
| catch (Exception ex) | |
| { | |
| logger.LogError(ex, "Error while closing PowerShell runspace pool '{PoolName}' during application stopping.", Name); | |
| } |
|
|
||
| # Collect bin/obj directories excluding anything under .git to be safe. | ||
| # Use -prune to avoid descending into matched directories after they are found. | ||
| mapfile -t TARGETS < <(find . -type d \( -name bin -o -name obj \) -not -path '*/.git/*' -prune -print) |
There was a problem hiding this comment.
This script uses mapfile, which requires Bash 4+. On macOS the system Bash is typically 3.2, so the script will fail unless users install a newer Bash. Consider rewriting this to avoid mapfile (e.g., a while IFS= read -r loop) or documenting/enforcing the Bash version requirement.
Suggested change
| mapfile -t TARGETS < <(find . -type d \( -name bin -o -name obj \) -not -path '*/.git/*' -prune -print) | |
| TARGETS=() | |
| while IFS= read -r dir; do | |
| TARGETS+=("$dir") | |
| done < <(find . -type d \( -name bin -o -name obj \) -not -path '*/.git/*' -prune -print) |
| "shell": { | ||
| "executable": "bash" | ||
| } | ||
| } |
There was a problem hiding this comment.
The new cleanup task hard-codes a Bash script but only configures a Bash shell override for Linux. On Windows runners/contributors this task will likely fail unless they’re using Git Bash/WSL. Consider adding a windows shell override (and/or an osx override) or documenting that the task requires Bash.
Suggested change
| } | |
| } | |
| }, | |
| "windows": { | |
| "options": { | |
| "shell": { | |
| "executable": "bash" | |
| } | |
| } | |
| }, | |
| "osx": { | |
| "options": { | |
| "shell": { | |
| "executable": "bash" | |
| } | |
| } |
Contributor
Minimum allowed line rate is |
aaronpowell
temporarily deployed
to
azure-artifacts
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrading the tests to use xUnit v3, which is what is used by aspire now too.
Close #961