KHR_gaussian_splatting proposal spec#72
KHR_gaussian_splatting proposal spec#72keyboardspecialist wants to merge 4 commits intocesium-nativefrom
Conversation
KHR_gaussian_splatting proposal spec
|
This looks fine to me @keyboardspecialist, and I'm happy to see it merged into the |
lilleyse
left a comment
lilleyse
left a comment
There was a problem hiding this comment.
@keyboardspecialist this looks pretty good. For now I just have comments on the technical aspects, but I think this could also use a style pass.
There was a problem hiding this comment.
Is quantizedPositionScale necessary? Usually with KHR_mesh_quantization the quantization scale is stored in the node transform.
There was a problem hiding this comment.
This table could be formatted more like the table in EXT_mesh_gpu_instancing
There was a problem hiding this comment.
Does COLOR_0 need to be restricted to VEC4 UNSIGNED_BYTE (normalized)? Someone may have VEC3 colors (no alpha) or floating point values. I feel like all the type/componentType combinations that glTF supports for vertex colors should be allowed here.
There was a problem hiding this comment.
Allowing float wouldn't be an issue I don't think. I don't know about omitting alpha. It's a core component of splatting. I would need to see the use case.
There was a problem hiding this comment.
True... I'm not sure if there's a use case for opaque splats.
There was a problem hiding this comment.
Given the conversations we have had with Adobe recently, I agree with @keyboardspecialist. I don't think that one could reasonably omit the alpha channel for splats. While we refer to the lowest-order spherical harmonic as diffuse color, that's not really what is going on there. The real view-dependent color is actually packed with the specular in the entire set of the spherical harmonics, and in that case the opacity is required.
Also, I just want to add that based on the conversations we've had with the community and Adobe, I'm guessing that fully supporting the higher-order spherical harmonics is going to need to be a major piece of our next steps in next year.
| } | ||
| ``` | ||
|
|
||
| ### Transforming Gaussian Splat Data for glTF |
There was a problem hiding this comment.
This section should be marked as non-normative.
There was a problem hiding this comment.
| As Gaussian splats are defined by a position color, rotation and scale, we both map to existing attributes and define new ones. These are attached to a primitive by defining the `extensions.KHR_gaussian_splatting` property on that mesh. | |
| As Gaussian splats are defined by a position color, rotation and scale, we both map to existing attributes and define new ones. These are attached to a primitive by defining the `extensions.KHR_gaussian_splatting` property on that primitive. |
There was a problem hiding this comment.
KHR_mesh_quantization doesn't support half float... and unsigned int doesn't provide any space savings. Wouldn't unsigned short or unsigned byte be more appropriate depending on your target quantization bits?
There was a problem hiding this comment.
You're correct. It's actually currently using unsigned short.
| | meshopt configuration | value | | ||
| | --- | --- | | ||
| | Mode | Attributes | | ||
| | Filter | None | |
There was a problem hiding this comment.
The glTF keys/values should probably be used in these tables, e.g.
| | meshopt configuration | value | | |
| | --- | --- | | |
| | Mode | Attributes | | |
| | Filter | None | | |
| | meshopt configuration | value | | |
| | --- | --- | | |
| | `mode` | `"ATTRIBUTES"` | | |
| | `filter` | `"NONE"` | |
|
Superseded by #81 |
4 participants
Clean branch with just KHR_gaussian_splatting proposal spec
Extension allows glTF files to treat point primitives as Gaussian splats.