ExecutionContext_NotCachedInSocketAsyncEventArgs - failure in Assert.True

[trylek]

Platform: Linux_musl x64 Release
Pipeline: runtime-libraries outerloop
Example run: https://dev.azure.com/dnceng/public/_build/results?buildId=482386&view=logs&j=9edfae9b-c909-5390-dc2d-2665643e0586
Proximate diagnostic info:

  Starting:    System.Net.Sockets.Tests (parallel test collections = on, max threads = 2)
    System.Net.Sockets.Tests.ExecutionContextFlowTest.ExecutionContext_NotCachedInSocketAsyncEventArgs [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/ExecutionContextFlowTest.cs(535,0): at System.Net.Sockets.Tests.ExecutionContextFlowTest.ExecutionContext_NotCachedInSocketAsyncEventArgs()
  Finished:    System.Net.Sockets.Tests
=== TEST EXECUTION SUMMARY ===
   System.Net.Sockets.Tests  Total: 515, Errors: 0, Failed: 1, Skipped: 0, Time: 91.496s

[v-haren]

failed again in job: runtime-libraries outerloop 20200615.2

failed test: System.Net.Sockets.Tests.ExecutionContextFlowTest.ExecutionContext_NotCachedInSocketAsyncEventArgs

Error message

Assert.True() Failure
Expected: True
Actual: False


Stack trace
   at System.Net.Sockets.Tests.ExecutionContextFlowTest.ExecutionContext_NotCachedInSocketAsyncEventArgs() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/ExecutionContextFlowTest.cs:line 535

[BruceForstall]

@karelz You marked this as "Future", but it's failing in the CI. Are you going to disable the test so we don't see this noise?

[karelz]

@BruceForstall what kind of noise do you see from this? We are actively monitoring test results - mainly official runs and address things affecting CI. This one did not pop for us yet.
Note: Networking tests are inherently difficult to make 100% passing - our current bar is 1 failure per week to investigate.

cc @alnikola

[BruceForstall]

@karelz This particular test I see has failed twice since May. Overall, it looks like Sysetm.Net has 28 failures in the last 30 days (after I ignored PostAsync_Cancel_CancellationTokenPassedToContent which had hundreds of failures).

Maybe we need a different model for running/monitoring these tests, to avoid the noise. E.g., maybe System.Net tests should run in their own Azure DevOps pipelines, separate from the rest of the tests, and our vendor teams that investigate CI system failures and ignore them, and your team can be responsible for monitoring. We can discuss offline if you wish.

[alnikola]

@BruceForstall I checked this test history from 1st February till 18th June and found only 2 failures in 2 different months which doesn't look disruptive and is far below our threshold.

Method	Started	Message
ExecutionContext_NotCachedInSocketAsyncEventArgs	2020-06-15 11:49:58.1910000	Assert.True() Failure\nExpected: True\nActual: False
ExecutionContext_NotCachedInSocketAsyncEventArgs	2020-05-03 12:07:29.5040000	Assert.True() Failure\r\nExpected: True\r\nActual: False

[VincentBu]

Failed in job runtime-libraries-coreclr outerloop 20210113.2

Failed test : System.Net.Sockets.Tests.ExecutionContextFlowTest.ExecutionContext_NotCachedInSocketAsyncEventArgs

net6.0-Linux-Release-x64-CoreCLR_release-(Fedora.32.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-32-helix-20200512010618-efb9f14

Error message:

Assert.True() Failure
Expected: True
Actual:   False


Stack trace
   at System.Net.Sockets.Tests.ExecutionContextFlowTest.ExecutionContext_NotCachedInSocketAsyncEventArgs() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/ExecutionContextFlowTest.cs:line 534