`verify` fails to pass with internal tasking system

[syoyo]

verify fails to pass when built without ray packats(-DEMBREE_TASKING_SYSTEM=INTERNAL), even on x86-64 platform.

$ ./verify 
                                                                        create_device ...
Embree Ray Tracing Kernels 3.7.0 (72275957d467242edd283d206261f2410c5eabae)
  Compiler  : CLANG 8.0.0 (tags/RELEASE_800/final)
  Build     : Debug 
  Platform  : Linux (64bit)
  CPU       : Unknown CPU (AuthenticAMD)
   Threads  : 32
   ISA      : XMM YMM SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 POPCNT AVX F16C RDRAND AVX2 FMA3 LZCNT BMI1 BMI2 
   Targets  : SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 AVX AVXI AVX2 
   MXCSR    : FTZ=1, DAZ=1
  Config
    Threads : default
    ISA     : XMM YMM SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 POPCNT AVX F16C RDRAND AVX2 FMA3 LZCNT BMI1 BMI2 
    Targets : SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 AVX AVXI AVX2  (supported)
              SSE2 SSE4.2 AVX AVX2 AVX512SKX  (compile time enabled)
    Features: intersection_filter 
    Tasking : internal_tasking_system 

 [PASSED]
                                                       fast_allocator_regression_test ... [PASSED]
                                                                cache_regression_test ... [PASSED]
                                                         parallel_for_regression_test ... [PASSED]
                                                      parallel_reduce_regression_test ... [PASSED]
                                                       parallel_prefix_sum_regression ... [PASSED]
                                                     parallel_for_for_regression_test ... [PASSED]
                                          parallel_for_for_prefix_sum_regression_test ... [PASSED]
                                                   parallel_partition_regression_test ... [PASSED]
                                                           RadixSortRegressionTestU32 ... [PASSED]
                                                           RadixSortRegressionTestU64 ... [PASSED]
                                                         parallel_set_regression_test ... [PASSED]
                                                         parallel_map_regression_test ... [PASSED]
                                                           parallel_filter_regression ... [PASSED]
                                                          barrier_sys_regression_test ... [PASSED]
                                                                SSE2.multiple_devices ... [PASSED]
                                                                      SSE2.types_test ... [PASSED]
                                                                      SSE2.get_bounds ...++++++ [PASSED]
                                                               SSE2.get_linear_bounds ...++++++ [PASSED]
                                                                   SSE2.get_user_data ... [PASSED]
                                                                   SSE2.buffer_stride ...++++++++ [PASSED]
                                                                     SSE2.empty_scene ...++++++++++ [PASSED]
                                                                  SSE2.empty_geometry ...++++++++++ [PASSED]
                                                                           SSE2.build ...++++++++++ [PASSED]
                                                          SSE2.overlapping_primitives ...++++++++++ [PASSED]
                                                             SSE2.new_delete_geometry ...............................................................................................................................+++.++ [PASSED]
                                                                SSE2.user_geometry_id ...+++++ [PASSED]
                                                         SSE2.enable_disable_geometry ...+++++ [PASSED]
                                                                          SSE2.update ...-----------------------------------------------------+--+++++---++---++---+++++++-+++-++----++++++++++++++++++++++++++++ [FAILED]
                                                              SSE2.build_garbage_geom ..................................................... [PASSED]
                                                           SSE2.interpolate.triangles ...++++++ [PASSED]
                                                                SSE2.interpolate.grid ...++++++ [PASSED]
                                                              SSE2.interpolate.subdiv ...++++++ [PASSED]
                                                                SSE2.interpolate.hair ...++++++ [PASSED]
                                                                    SSE2.triangle_hit ...---+----++--+-+-----++-++-+--+--+-+-++++---++-+-+-+-++-----++--++++--++--+++--+-----+-+-+---++++-----+----+-+---+++-+--- [FAILED]
                                                                        SSE2.quad_hit ...----------++--+-+-+---++++-+----++-+-++---+++++----+++----++-+-+--+-++----+-+-+----++++--+-++--+-+++--+-+-+-+---+-----+- [FAILED]
                                                             SSE2.intersection_filter ...-+-----+---+--+++----+--+------++--+-+---++---+++++-+-+--+---+---++-+------++--++++-++-----+++-+---+-+----+++--+++-+--++----+--+---+---+++---++-----+------++---+------+--+++---++--++---+-+++-----+++--+-++++++-++--+-+++---+--++++-++--+++-+-- [FAILED]
                                                                      SSE2.instancing ...---+verify: /home/syoyo/work/embree/tutorials/verify/verify.cpp:2813: virtual VerifyApplication::TestReturnValue embree::InstancingTest::run(embree::VerifyApplication *, bool): Assertion `passed' failed.

[syoyo]

It looks the reason is using INTERNAL tasking system. On x86 linux, verify passes when using TBB as a tasking backend.

[maikschulze]

For Windows x64, I can confirm that the "verify" test fails with the INTERNAL tasking system in 'SSE2.update' and further tests afterwards when setting EMBREE_RAY_PACKETS=OFF. With EMBREE_RAY_PACKETS=ON these tests pass. I did not wait until completion of verify.

[syoyo]

@maikschulze thanks for the report! So it looks its an (known?) issue of original embree. At least we don't see any significant visual error with INTERNAL build so we can lower the priority of this issue.

[maikschulze]

Yes, @syoyo, I can confirm that this test fails in the original Embree 3.8.0 as well. I compiled with EMBREE_RAY_PACKETS=OFF in debug mode and was able to reproduce.
Note it's best to repro with these commandline arguments:
--run .*SSE2.update.* --sequential

I will dig a bit and create an issue in the original Embree project.

[maikschulze]

At first sight, it appears to be an easy fix. rtcIntersect4 seems to omit writing the hit information into the rayhit 4-packet in the "emulation".

#if !defined(EMBREE_RAY_PACKETS)
    Ray4* ray4 = (Ray4*) rayhit;
    for (size_t i=0; i<4; i++) {
      if (!valid[i]) continue;
      RayHit ray1; ray4->get(i,ray1);
      scene->intersectors.intersect((RTCRayHit&)ray1,&context);
      ray4->set(i,ray1); // ISSUE: calls RayK<K>::set instead of RayHitK<K>::set
    }
#else

[maikschulze]

I've been able to fix this and will inform Intel.

After fixing the areas for the '!EMBREE_RAY_PACKETS' code path in rtcIntersect4/8/16 and rtcOccluded4/8/16 and get a passing test:

SSE2.update ...++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [PASSED]

Tests passed: 520
Tests failed: 0
Tests failed and ignored: 0

[syoyo]

@maikschulze Awesome!

I've been able to fix this and will inform Intel.

I will backport your fix once you report it(create PR?) to intel embree repo.

[maikschulze]

@syoyo , thanks for raising the issue in the first place!
I've created the pull request here: RenderKit/embree#275

[syoyo]

@maikschulze Awesome!

[maikschulze]

Hi @syoyo , I've made further fixes to the verify executable and made a second pull request for them: RenderKit/embree#276

After applying both pull requests, all tests should pass on x64. Could you please verify this? Could you also test on arm64? Fingers crossed all tests pass there as well.

[syoyo]

@maikschulze OK. Merging PR itself to embree-aarch64 master and neon-fix gets conflicts. I'm planning to cherry pick changes.

[syoyo]

Hmm... cherry-pick didn't work.

I need fist merge neon-fix to master, then keep up embree-aarch64 master with intel embree v3.8.0

[syoyo]

@maikschulze FYI, I have added running NEON tests(reuses SSE2 tests) in verify using v3.8.0 + neon-fix branch but got seg faults even with EMBREE_RAY_PACKETS=On.

#22