Rendering Engine: Fix alpha support

[Popov72]

I took Gary's WIP PR (#15573) as a base, but decided to go another way: keep all the existing code and simply honor the transparencyMode value when it's set.

This is because the existing code is quite complicated and has a lot of interconnections. I felt that trying to simplify it was too complicated and error-prone, so I made changes to simply use the transparencyMode parameter when set and keep the existing code-path otherwise.

At some point, we could deprecate the existing code and keep only the code related to transparencyMode. In addition, I think it's easier to document the breaking change:

• if you give a non-null value to transparencyMode, it will always be honored, whatever values you give to other properties (like mesh visibility, material alpha, etc)
• If transparencyMode is null, no change is made to the existing behavior, which is fully compatible with the past.

Other changes (from Gary's PR):

• _shouldTurnAlphaTestOn has been renamed to needAlphaTestingForMesh. This is not a breaking change, since _shouldTurnAlphaTestOn was protected, and more in line with the existing needAlphaBlendingForMesh method.
• All occurrences of needAlphaTesting have been replaced by needAlphaTestingForMesh: the behavior might change because of this, but it's not a breaking change but a bug fix.

Regarding failed visualization tests (3):

• "NME loop block". It didn't fail because of the changes but because of timing issues (a node material was loaded in parallel to the test - I now wait for the material to load before returning from the createScene function).
• "Visibility" and "Prepass SSAO + visibility" are in fact the same test. They are expected to fail, as they set transparencyMode=OPAQUE and define certain properties that allow blending in the current code path, whereas with the PR, the material is no longer transparent. To correct the tests, I set transparencyMode=ALPHABLEND and useAlphaFromAlbedoTexture=false. Indeed, when useAlphaFromAlbedoTexture=true (as was the case in the test), the alpha of the albedo texture is taken into account when transparencyMode=ALPHABLEND (as expected), but not in the actual code path when transparencyMode=OPAQUE. So the images were different if I didn't set useAlphaFromAlbedoTexture to false.

cc @sebavan and @bghgary

[bjsplat]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[bjsplat]

Snapshot stored with reference name:
refs/pull/16144/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/16144/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/16144/merge
https://gui.babylonjs.com/?snapshot=refs/pull/16144/merge
https://nme.babylonjs.com/?snapshot=refs/pull/16144/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/16144/merge#BCU1XR#0

[bjsplat]

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/testResults/webgl2playwright/index.html

[bghgary]

@Popov72 Can you make sure your changes fixes the problem described here? #15504

[bjsplat]

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/testResults/webgpuplaywright/index.html

[sebavan]

@RaananW is there an easy way to flag some functions to error out if they are used within the repo itself ? so that we could be sure to not make any mistake by using them in the future.

[bghgary]

@Popov72 Sorry, one more issue to check: #15557

[Popov72]

@Popov72 Can you make sure your changes fixes the problem described here? #15504

It doesn't, but it seems the problem is with NVidia GPU. Even Khronos Sample Viewer doesn't display it correctly on my computer (RTX 3080), contrary to what Ed Mackey says in KhronosGroup/glTF-Sample-Assets#146:

In Babylon with the PR:

I will create an issue in the Chromium repo about this bug.

[bjsplat]

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/testResults/webgpuplaywright/index.html

[bjsplat]

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/testResults/webgl2playwright/index.html

[Popov72]

@Popov72 Sorry, one more issue to check: #15557

This one does work with the PR.

[bghgary]

It doesn't, but it seems the problem is with NVidia GPU. Even Khronos Sample Viewer doesn't display it correctly on my computer (RTX 3080), contrary to what Ed Mackey says in KhronosGroup/glTF-Sample-Assets#146:

Sorry, I wasn't expecting this change to fix the middle two issues. I'm only worried about the first one. I don't think it makes sense for the middle two in practice. I would not expect assets that use floating point boundaries to work.

[RaananW]

@RaananW is there an easy way to flag some functions to error out if they are used within the repo itself ? so that we could be sure to not make any mistake by using them in the future.

Any example in this PR? So I will understand the use case?

[sebavan]

@RaananW is there an easy way to flag some functions to error out if they are used within the repo itself ? so that we could be sure to not make any mistake by using them in the future.

Any example in this PR? So I will understand the use case?

Material needAlphaBlending() and needAlphaTesting() are only kept for back compat but should never be called from within the framework as we need to rely on their replacements needAlphaBlendingForMesh and needAlphaTestingForMesh.

So we want to keep the old ones in but prevent their use internally.

[RaananW]

Material needAlphaBlending() and needAlphaTesting() are only kept for back compat but should never be called from within the framework as we need to rely on their replacements needAlphaBlendingForMesh and needAlphaTestingForMesh.

So we want to keep the old ones in but prevent their use internally.

Set them to be deprecated. We can then enable a no-deprecated API rule on eslint.

[sebavan]

Material needAlphaBlending() and needAlphaTesting() are only kept for back compat but should never be called from within the framework as we need to rely on their replacements needAlphaBlendingForMesh and needAlphaTestingForMesh.
So we want to keep the old ones in but prevent their use internally.

Set them to be deprecated. We can then enable a no-deprecated API rule on eslint.

Perfect !!! cc @Popov72

[bjsplat]

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/testResults/webgpuplaywright/index.html

[bjsplat]

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16144/merge/testResults/webgl2playwright/index.html

[chase-moskal]

hello, i thought i should post here to help anybody else who encounters the same issue i did

i found that upgrading from babylon 7.47.2 to 7.47.3 caused my mesh.visibility = 0.5 code to stop working, we used it to make walls fade away, and then it stopped working, and i narrowed it down to this babylon version range, and searching for changes related to the keyword visibility, i found this PR, and i see that it's marked as a breaking change involving transparency modes.

• i implemented the following fix into my codebase
  fix: wall fading, babylon breaking change
  benevolent-games/rogue-crusade@cbb52d0

basically, i just had to make sure the relevant material was using alpha blending mode, like this:

mesh.material.transparencyMode = Material.MATERIAL_ALPHABLEND

this fixed the issue for me 👍

i just wanted to share in case it helps somebody searching with the same issue, i'm sure the breaking change is worthwhile for these improvements to alpha support.

i do wish babylon's versioning followed semver rules, so that during an upgrade it was more obvious when a breaking change might occur, it took me awhile to diagnose that this bug was caused by a babylon breaking change from a routine upgrade from weeks ago that was the culprit.

i'll have to watch the changelog more carefully -- also it would be nice if breaking changes were associated with some kind of instruction or hint as to what downstream bablyon users will be likely to need to change and how, i basically made a guess

[Popov72]

You're right, I should have updated the "Breaking change" doc. Here's the PR:

BabylonJS/Documentation#1276