Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .github/agents/test-developer.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,13 +64,73 @@ public void ClassName_MethodUnderTest_Scenario_ExpectedBehavior()
- Failure-testing and error handling scenarios
- Verifying internal behavior beyond requirement scope

### Test Source Filters

Test links in `requirements.yaml` can include a source filter prefix to restrict which test results count as
evidence. These filters are critical for platform, simulator, and framework requirements - **do not remove them**.

- `ghdl@TestName` - proves the test passed using the GHDL simulator
- `nvc@TestName` - proves the test passed using the NVC simulator
- `windows@TestName` - proves the test passed on a Windows platform
- `ubuntu@TestName` - proves the test passed on a Linux (Ubuntu) platform
- `dotnet8.x@TestName` - proves the self-validation test ran on a machine with .NET 8.x runtime
- `dotnet9.x@TestName` - proves the self-validation test ran on a machine with .NET 9.x runtime
- `dotnet10.x@TestName` - proves the self-validation test ran on a machine with .NET 10.x runtime

Removing a source filter means a test result from any environment can satisfy the requirement, which invalidates
the evidence-based proof that the tool works on a specific platform, simulator, or framework.

### VHDLTest-Specific

- **NOT self-validation tests** - those are handled by Software Developer Agent
- Unit tests live in `test/` directory
- Use MSTest V4 testing framework
- Follow existing naming conventions in the test suite

### MSTest V4 Best Practices

Common anti-patterns to avoid (not exhaustive):

1. **Avoid Assertions in Catch Blocks (MSTEST0058)** - Instead of wrapping code in try/catch and asserting in the
catch block, use `Assert.ThrowsExactly<T>()`:

```csharp
var ex = Assert.ThrowsExactly<ArgumentNullException>(() => SomeWork());
Assert.Contains("Some message", ex.Message);
```

2. **Avoid using Assert.IsTrue / Assert.IsFalse for equality checks** - Use `Assert.AreEqual` /
`Assert.AreNotEqual` instead, as it provides better failure messages:

```csharp
// ❌ Bad: Assert.IsTrue(result == expected);
// ✅ Good: Assert.AreEqual(expected, result);
```

3. **Avoid non-public test classes and methods** - Test classes and `[TestMethod]` methods must be `public` or
they will be silently ignored:

```csharp
// ❌ Bad: internal class MyTests
// ✅ Good: public class MyTests
```

4. **Avoid Assert.IsTrue(collection.Count == N)** - Use `Assert.HasCount` for count assertions:

```csharp
// ❌ Bad: Assert.IsTrue(collection.Count == 3);
// ✅ Good: Assert.HasCount(3, collection);
```

5. **Avoid Assert.IsTrue for string prefix checks** - Use `Assert.StartsWith` instead of wrapping
`string.StartsWith` in `Assert.IsTrue`, as it produces clearer failure messages that show the expected prefix
and actual value:

```csharp
// ❌ Bad: Assert.IsTrue(value.StartsWith("prefix"));
// ✅ Good: Assert.StartsWith("prefix", value);
```

## Defer To

- **Requirements Agent**: For test strategy and coverage requirements
Expand Down
3 changes: 3 additions & 0 deletions src/DEMAConsulting.VHDLTest/Context.cs
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,9 @@ public void WriteError(string? message)
/// <exception cref="InvalidOperationException">On invalid arguments</exception>
public static Context Create(string[] args)
{
// Validate input
ArgumentNullException.ThrowIfNull(args);

// Process the arguments
var help = false;
var version = false;
Expand Down
24 changes: 18 additions & 6 deletions src/DEMAConsulting.VHDLTest/DEMAConsulting.VHDLTest.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<LangVersion>latest</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>

<!-- NuGet Tool Package Configuration -->
<PackAsTool>True</PackAsTool>
<PackAsTool>true</PackAsTool>
<ToolCommandName>vhdltest</ToolCommandName>
<PackageId>DemaConsulting.VHDLTest</PackageId>
<Version>0.0.0</Version>
Expand All @@ -26,15 +26,15 @@
<Product>DemaConsulting.VHDLTest</Product>

<!-- Symbol Package Configuration -->
<IncludeSymbols>True</IncludeSymbols>
<IncludeSymbols>true</IncludeSymbols>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<EmbedUntrackedSources>true</EmbedUntrackedSources>
<ContinuousIntegrationBuild Condition="'$(GITHUB_ACTIONS)' == 'true'">true</ContinuousIntegrationBuild>

<!-- Code Quality Configuration -->
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<GenerateDocumentationFile>True</GenerateDocumentationFile>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
<EnableNETAnalyzers>true</EnableNETAnalyzers>
<AnalysisLevel>latest</AnalysisLevel>
Expand All @@ -48,14 +48,26 @@

<ItemGroup>
<PackageReference Include="DemaConsulting.TestResults" Version="1.5.0" />
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="10.0.103" PrivateAssets="All" />
<!-- PrivateAssets="All" prevents these build-time-only packages from becoming transitive dependencies -->
<PackageReference Include="Microsoft.Sbom.Targets" Version="4.1.5" PrivateAssets="All" />
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="10.0.103" PrivateAssets="All" />
<PackageReference Include="YamlDotNet" Version="16.3.0" />
<!--
Analyzer and source-only packages require both PrivateAssets and IncludeAssets:
PrivateAssets="all" - prevents these build-time assets from flowing to consumers
IncludeAssets - explicitly enables contentfiles (source injection) and
analyzers/buildtransitive (Roslyn analyzers at compile time)
-->
<PackageReference Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.103">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="SonarAnalyzer.CSharp" Version="10.19.0.132793">
<PackageReference Include="SonarAnalyzer.CSharp" Version="10.20.0.135146">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<!-- Polyfill is a source-only package; contentfiles delivers the polyfill source into this project -->
<PackageReference Include="Polyfill" Version="9.12.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
Expand Down
3 changes: 3 additions & 0 deletions src/DEMAConsulting.VHDLTest/Validation.cs
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@
/// <param name="context">Program context</param>
public static void Run(Context context)
{
// Validate input
ArgumentNullException.ThrowIfNull(context);

// Write validation header
context.WriteLine(
$"""
Expand Down Expand Up @@ -186,7 +189,7 @@
/// <param name="exitCode">Program exit-code</param>
/// <param name="output">Output text</param>
/// <param name="succeeded">True if test succeeded</param>
private static void ReportTestResult(

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (windows-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (windows-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (windows-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (windows-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (windows-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (windows-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (ubuntu-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (ubuntu-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (ubuntu-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (ubuntu-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (ubuntu-latest)

Method has 8 parameters, which is greater than the 7 authorized.

Check warning on line 192 in src/DEMAConsulting.VHDLTest/Validation.cs

View workflow job for this annotation

GitHub Actions / Build and Test / build (ubuntu-latest)

Method has 8 parameters, which is greater than the 7 authorized.
Context context,
TestResults results,
string testName,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,39 +11,47 @@

<!-- Code Quality Configuration -->
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<GenerateDocumentationFile>True</GenerateDocumentationFile>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
<EnableNETAnalyzers>true</EnableNETAnalyzers>
<AnalysisLevel>latest</AnalysisLevel>
</PropertyGroup>

<!-- Implicit Usings -->
<ItemGroup>
<Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
<Using Include="Polyfills" />
</ItemGroup>

<!-- Test Framework Dependencies -->
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="8.0.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.3.0" />
<PackageReference Include="MSTest.TestAdapter" Version="4.1.0" />
<PackageReference Include="MSTest.TestFramework" Version="4.1.0" />
</ItemGroup>

<!-- Code Analysis Dependencies -->
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.103">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="SonarAnalyzer.CSharp" Version="10.19.0.132793">
<PackageReference Include="SonarAnalyzer.CSharp" Version="10.20.0.135146">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>

<!-- Project References -->
<ItemGroup>
<ProjectReference Include="..\..\src\DEMAConsulting.VHDLTest\DEMAConsulting.VHDLTest.csproj">
<Private>True</Private>
<CopyLocalSatelliteAssemblies>True</CopyLocalSatelliteAssemblies>
</ProjectReference>
</ItemGroup>

<ItemGroup>
<Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
</ItemGroup>

</Project>