diff --git a/extensions/2.0/Vendor/EXT_mesh_features/README.md b/extensions/2.0/Vendor/EXT_mesh_features/README.md index 5dcd8fde7a..b9c8ebd327 100644 --- a/extensions/2.0/Vendor/EXT_mesh_features/README.md +++ b/extensions/2.0/Vendor/EXT_mesh_features/README.md @@ -8,6 +8,7 @@ * Sean Lilley, Cesium * Sam Suhag, Cesium * Patrick Cozzi, Cesium +* Don McCurdy, Independent * Bao Tran, Cesium * Samuel Vargas, Cesium @@ -21,100 +22,113 @@ Draft Written against the glTF 2.0 specification. -Adds new functionality to the [`EXT_mesh_gpu_instancing` extension](../../EXT_mesh_gpu_instancing). +Optionally, this extension may be used in conjunction with [`EXT_mesh_gpu_instancing`](../../EXT_mesh_gpu_instancing). When used together, certain GPU instance attributes defined by `EXT_mesh_gpu_instancing` are used as [instance feature IDs](#feature-id-by-gpu-instance). -## Optional vs. Required - -This extension is optional, meaning it should be placed in the `extensionsUsed` list, but not in the `extensionsRequired` list. - - -## Contents +## Table of Contents - [Overview](#overview) -- [Feature Identification](#feature-identification) - - [Feature ID Vertex Attributes](#feature-id-vertex-attributes) - - [Feature ID Accessors](#feature-id-accessors) - - [Implicit Feature IDs](#implicit-feature-ids) - - [Feature ID Textures](#feature-id-textures) - - [Feature ID Instance Attributes](#feature-id-instance-attributes) +- [Feature IDs](#feature-ids) + - [Overview](#overview-1) + - [Feature ID by Vertex](#feature-id-by-vertex) + - [Vertex Attribute](#vertex-attribute) + - [Implicit Vertex Attribute](#implicit-vertex-attribute) + - [Feature ID by Texture Coordinates](#feature-id-by-texture-coordinates) + - [Feature ID by GPU Instance](#feature-id-by-gpu-instance) + - [Specifying Feature IDs](#specifying-feature-ids) - [Feature Properties](#feature-properties) - - [Schemas](#schemas) + - [Overview](#overview-2) + - [Schema Definitions](#schema-definitions) + - [Overview](#overview-3) + - [Schema](#schema) + - [Class](#class) + - [Class Property](#class-property) + - [Enum](#enum) + - [Enum Values](#enum-values) - [Property Tables](#property-tables) - [Property Textures](#property-textures) - [Binary Data Storage](#binary-data-storage) +- [Optional vs. Required](#optional-vs-required) +- [Schema](#schema-1) - [Examples](#examples) -- [Schema](#schema) - [Revision History](#revision-history) ## Overview -A **feature** is an entity that has both geometry and associated properties. In Geographic Information Systems (GIS) a feature is an entity such as a point, polyline, or polygon that represents some element on a map. In another domain like CAD/BIM a feature might be a component of a design model. A feature could also be a 3D building in a city, a tree in a forest, a sample point in a weather model, or a patch of texels on a 3D model. +This extension defines a means of storing structured metadata associated with geometry and subcomponents of geometry within a glTF 2.0 asset. -This extension allows batching features for efficient streaming to a client for rendering and interaction. Efficiency comes from transferring multiple features in the same glTF and rendering them in the least number of draw calls necessary. +In most realtime 3D contexts, performance requirements demand minimizing the number of nodes and meshes in an asset. These requirements compete with interactivity, as applications may wish to merge static objects while still supporting interaction or inspection on those objects. Common performance optimizations for GPU rendering — like merging geometry or GPU instancing to reduce CPU overhead — may destroy references to distinct objects, their attributes, and their behaviors. -Feature IDs enable individual features to be identified and updated at runtime. For example, a selected feature could be shown/hidden, or highlighted a different color. Feature IDs may be assigned on a per-vertex, per-texel, or per-instance basis. +By defining a representation of conceptual objects ("features") distinct from rendered geometry, and a means of associating structured metadata ("properties") with those features, this extension allows applications to preserve important details of 3D assets for inspection and interaction without compromising runtime performance and draw calls. -Feature IDs may be used to access associated properties, such as passing a building's ID to get its address. Feature properties are stored in a compact binary tabular format described in the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata). +Concepts and terminology used throughout this document refer to the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/blob/3d-tiles-next/specification/Metadata/README.md), which should be considered a normative reference for definitions and requirements. This document provides inline definitions of terms where appropriate. -![Building Example](figures/feature-table-buildings.png) +See [Examples](#examples) for a more detailed list of use cases for this extension. -In the image above, a glTF consists of two houses batched together into a single primitive. A feature ID attribute on the primitive indicates that all of the vertices making up the first house have a feature ID of 0, while all vertices making up the second house have the feature ID 1. The feature ID is then used to access the building's properties from the property table. +> **Disambiguation:** glTF has other methods of storing details that could similarly be described as metadata or properties, including [`KHR_xmp_json_ld`](https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_xmp_json_ld), Extras, and Extensions. While those methods associate data with discrete objects in a glTF asset — nodes, materials, etc. — `EXT_mesh_features` is uniquely suited for properties of more granular conceptual features as small as individual vertices or texels. -Feature properties may also be stored directly in textures. This is especially useful when texture mapping high frequency data, such as material properties, to less detailed 3D surfaces. Property textures enable new styling and analytics capabilities, and complement glTF PBR textures. +## Feature IDs -See [Examples](#examples) for a full list of use cases for this extension. +### Overview -## Feature Identification +A **feature** is a conceptual object associated with both geometry and properties. Similar concepts exist in various industries and domains. In Geographic Information Systems (GIS) a feature is an entity such as a point, polyline, or polygon that represents some element on a map. In another domain like CAD/BIM a feature might be a component of a design model, such as a pipe. A feature could also be a 3D building in a city, a tree in a forest, a sample point in a weather model, or a patch of texels on a 3D asset. -Features in a glTF primitive are identified in three ways: +Features are identified within a 3D asset by **Feature IDs** (unique identifiers) associated with parts of the asset in one of three ways: -* Per-vertex using a vertex attribute -* Per-texel using a glTF texture -* Per-instance using an instance attribute with the [`EXT_mesh_gpu_instancing` extension](../../EXT_mesh_gpu_instancing) +* **Feature ID by Vertex:** Per-vertex ID, in a vertex attribute or derived from the vertex index +* **Feature ID by Texture Coordinates:** Per-texel ID, in a texture channel +* **Feature ID by GPU Instance:** Per-instance ID, in an instance attribute (requires [`EXT_mesh_gpu_instancing`](../../EXT_mesh_gpu_instancing)) -Metadata Access +### Feature ID by Vertex -### Feature ID Vertex Attributes +#### Vertex Attribute -#### Feature ID Accessors +*Defined in [featureIdAttribute.schema.json](./schema/featureIdAttribute.schema.json).* -The most straightforward method for defining feature IDs is to store them in a glTF vertex attribute. Feature ID attributes must follow the naming convention `FEATURE_ID_X` where `X` is a non-negative integer. The first feature ID attribute is `FEATURE_ID_0`, the second `FEATURE_ID_1`, and so on. +Per-vertex feature IDs may be defined explicitly in a vertex attribute accessor. -Feature IDs are whole numbers in the range `[0, count - 1]` (inclusive), where `count` is the total number of features in the property table. Values outside this range indicate that no feature is associated. +Names of feature ID attribute semantics follow the naming convention `FEATURE_ID_n` where `n` must start with 0 and continue with consecutive positive integers: `FEATURE_ID_0`, `FEATURE_ID_1`, etc. Indices must not use leading zeroes to pad the number of digits (e.g., `FEATURE_ID_01` is not allowed). -The attribute's accessor `type` must be `"SCALAR"` and `normalized` must be false. There is no restriction on `componentType`. +Values of feature IDs are non-negative integers in the range `[0, count - 1]` (inclusive), where `count` is the total number of features in the [property table](#property-tables). Values outside this range indicate that no feature is associated. -> **Implementation Note:** since glTF accessors do not support `UNSIGNED_INT` types for 32-bit integers, `FLOAT` may be used instead. This allows for integer feature IDs up to 2²⁴. For smaller ranges of feature IDs, `UNSIGNED_BYTE` or `UNSIGNED_SHORT` can still be used. Note that this requires aligning each feature ID to 4-byte boundaries to adhere to glTF's alignment rules. +The attribute's accessor `type` must be `"SCALAR"` and `normalized` must be false. The accessor's `componentType` is not restricted. -![Property Table](figures/feature-table.png) +> **Implementation Note:** since glTF accessors do not support `UNSIGNED_INT` types for 32-bit integers, `FLOAT` may be used instead allowing integer feature IDs up to 224. For smaller ranges of feature IDs, `UNSIGNED_BYTE` or `UNSIGNED_SHORT` should be used. As with other vertex attributes, each element of a feature ID accessor must align to 4-byte boundaries. -```jsonc -{ - "primitives": [ - { - "attributes": { - "POSITION": 0, - "FEATURE_ID_0": 1 - }, - "indices": 2, - "mode": 4, - "extensions": { - "EXT_mesh_features": { - "propertyTables": [0], - "featureIds": [{"attribute": 0}] - } - } - } - ] -} -``` +> **Example:** A primitive defines two quads, where each quad is a distinct feature. The quads are composed of four vertices, distinguished by different `FEATURE_ID_0` vertex attribute values. Each feature is associated with "Name", "Year", and "Coordinates" values in a [property table](#property-tables). +> +> Note that `"attribute": 0` refers to `FEATURE_ID_0`. +> +> ![Property Table](figures/feature-table.svg) +> +> ```jsonc +> { +> "primitives": [ +> { +> "attributes": { +> "POSITION": 0, +> "FEATURE_ID_0": 1 +> }, +> "indices": 2, +> "mode": 4, +> "extensions": { +> "EXT_mesh_features": { +> "propertyTables": [0], +> "featureIds": [{"attribute": 0}] +> } +> } +> } +> ] +> } +> ``` -#### Implicit Feature IDs +#### Implicit Vertex Attribute -In some cases it is possible to define feature IDs implicitly, such as when all vertices in a primitive have the same feature ID or when each vertex in a primitive has a different feature ID. +*Defined in [featureIdAttribute.schema.json](./schema/featureIdAttribute.schema.json).* -This is accomplished by using `offset` and `repeat`. +Per-vertex feature IDs may also be defined implicitly, as a function of vertex index within the primitive. Implicit feature IDs reduce storage costs in several common cases, such as when all vertices in a primitive share the same feature ID, or each sequential group of `N` vertices (e.g. each triangle face) share the same feature ID. + +Implicit feature IDs are a monotonically increasing function of the vertex index, configured by `offset` and `repeat` parameters. * `offset` specifies the initial value for the vertex feature ID range. The default is `0`. * `repeat`, if defined, specifies the number of vertices for which to repeat each feature ID before incrementing the ID by 1. If `repeat` is undefined, the feature ID for all vertices is `offset`. @@ -127,263 +141,509 @@ For example * If `offset` is 2 and `repeat` is 3, the feature IDs are `[2, 2, 2, 3, 3, 3, 4, 4, 4, ...]` * If `offset` is 2 and `repeat` is undefined, the feature IDs are `[2, 2, 2, ...]` -`offset` and `repeat` must be omitted when `attribute` is used. These two methods of assigning feature IDs are mutually exclusive. - -Placemarks - -```jsonc -{ - "primitives": [ - { - "attributes": { - "POSITION": 0 - }, - "mode": 0, - "extensions": { - "EXT_mesh_features": { - "propertyTables": [0], - "featureIds": [{"offset": 0, "repeat": 1}] - } - } - } - ] -} -``` -### Feature ID Textures - -Feature ID textures classify the pixels of an image into different features. Some examples include image segmentation or marking regions on a map. - -Often per-texel feature IDs provide finer granularity than per-vertex feature IDs as in the example below. - -Feature ID Texture - -```jsonc -{ - "primitives": [ - { - "attributes": { - "POSITION": 0, - "TEXCOORD_0": 1 - }, - "indices": 2, - "material": 0, - "extensions": { - "EXT_mesh_features": { - "propertyTables": [0], - "featureIds": [ - {"index": 0, "texCoord": 0, "channel": 0} - ] - } - } - } - ] -} -``` - -The `featureId` entry for a feature ID texture extends the glTF [`textureInfo`](../../../../../specification/2.0/schema/textureInfo.schema.json) object. Each `channel` must be a non-negative integer corresponding to a channel of the source texture. Channels of an `RGBA` texture are numbered 0–3 respectively, although specialized texture formats may allow additional channels. Feature IDs are whole numbers in the range `[0, count - 1]` (inclusive), stored in linear space, where `count` is the total number of features in the property table. Values outside this range indicate that no feature is associated. +> **Example:** Each point in the point cloud below represents a distinct feature. Points are identified by feature IDs 0–5. Each point is associated "Name" and "Elevation" values a [property table](#property-tables). +> +> Placemarks +> +> ```jsonc +> { +> "primitives": [ +> { +> "attributes": { +> "POSITION": 0 +> }, +> "mode": 0, +> "extensions": { +> "EXT_mesh_features": { +> "propertyTables": [0], +> "featureIds": [{"offset": 0, "repeat": 1}] +> } +> } +> } +> ] +> } +> ``` + +### Feature ID by Texture Coordinates + +*Defined in [featureIdTexture.schema.json](./schema/featureIdTexture.schema.json).* + +Feature ID textures classify the pixels of an image into different features. Some use cases include image segmentation or marking regions on a map. Often per-texel feature IDs provide finer granularity than per-vertex feature IDs, as in the example below. + +> **Example:** A building facade, represented by a single quad. The primitive's `baseColorTexture` displays the visible appearance of the building, and its feature ID texture identifies regions of the quad (walls, door, roof, window) as distinct features. Both textures use the same texture coordinates, `TEXCOORD_0`, in this example. Each feature is associated with "Component" and "Year Built" values in a [property table](#property-tables). +> +> Feature ID Texture +> +> ```jsonc +> { +> "primitives": [ +> { +> "attributes": { +> "POSITION": 0, +> "TEXCOORD_0": 1 +> }, +> "indices": 2, +> "material": 0, +> "extensions": { +> "EXT_mesh_features": { +> "propertyTables": [0], +> "featureIds": [ +> {"index": 0, "texCoord": 0, "channel": 0} +> ] +> } +> } +> } +> ] +> } +> ``` + +A `featureId` pointing to a feature ID texture extends the glTF [`textureInfo`](../../../../../specification/2.0/schema/textureInfo.schema.json) object. Its `channel` must be a non-negative integer corresponding to a channel of the source texture. Channels of an `RGBA` texture are numbered 0–3 respectively, although specialized texture formats may allow additional channels. + +Feature ID values stored in a texture are non-negative integers in the range `[0, count - 1]` (inclusive), stored in linear space, where `count` is the total number of features in the property table. Values outside this range indicate that no feature is associated. Texture filtering must be `9728` (NEAREST), or undefined, for any texture object referenced as a feature ID texture. -### Feature ID Instance Attributes - -Feature IDs may also be assigned to individual instances when using the [`EXT_mesh_gpu_instancing` extension](../../EXT_mesh_gpu_instancing). This works the same way as assigning feature IDs to vertices. Feature IDs may be stored in accessors or generated implicitly. Nodes with `EXT_mesh_features` must also define an `EXT_mesh_gpu_instancing` extension providing feature ID instance attributes, and are invalid without this dependency. - -```jsonc -{ - "nodes": [ - { - "mesh": 0, - "extensions": { - "EXT_mesh_gpu_instancing": { - "attributes": { - "TRANSLATION": 0, - "ROTATION": 1, - "SCALE": 2, - "FEATURE_ID_0": 3 - }, - }, - "EXT_mesh_features": { - "propertyTables": [0], - "featureIds": [{"attribute": 0}] - } - } - } - ] -} -``` +### Feature ID by GPU Instance + +*Defined in [featureIdAttribute.schema.json](./schema/featureIdAttribute.schema.json).* + +Feature IDs may also be assigned to individual GPU instances when using the [`EXT_mesh_gpu_instancing` extension](../../EXT_mesh_gpu_instancing). Similar to per-vertex IDs, per-instance IDs are stored in instance attributes or generated implicitly by instance index. Nodes with `EXT_mesh_features` must also define an `EXT_mesh_gpu_instancing` extension, and are invalid without this dependency. + +> **Example:** A node defining instances of mesh `0`, with each instance having a feature ID in the `FEATURE_ID_0` instance attribute. +> +> ```jsonc +> { +> "nodes": [ +> { +> "mesh": 0, +> "extensions": { +> "EXT_mesh_gpu_instancing": { +> "attributes": { +> "TRANSLATION": 0, +> "ROTATION": 1, +> "SCALE": 2, +> "FEATURE_ID_0": 3 +> }, +> }, +> "EXT_mesh_features": { +> "propertyTables": [0], +> "featureIds": [{"attribute": 0}] +> } +> } +> } +> ] +> } +> ``` + +### Specifying Feature IDs + +A primitive or node may specify multiple feature IDs using one or more of the methods above. With multiple feature IDs, an asset might (for example) identify features at different levels of abstraction: per-vertex feature IDs for individual buildings, and per-texel feature IDs for parts of each building, with each level of abstraction having its own properties. + +Each feature ID definition may include only a single source, so the following are mutually exclusive: + +- `featureId.attribute` +- `featureId.offset` and `featureId.repeat` +- `featureId.index` + +The `featureIds` and `propertyTables` arrays must have the same length, with the feature ID definition at index `i` corresponding to the property table at the same index. Each `featureIds:propertyTable` pair must be unique, but individual feature IDs and property tables may be repeated within a primitive or node. + +Empty feature IDs (e.g. `{}`) are disallowed — a feature ID must explicitly set at least one property. + +> **Example:** Multiple property tables and feature IDs may be defined on a single primitive. +> +> ```jsonc +> // Primitive: +> "extensions": { +> "EXT_mesh_features": { +> "propertyTables": [0, 1, 2], +> "featureIds": [ +> {"attribute": 0}, +> {"offset": 1}, +> {"index": 0, "texCoord": 0, "channel": 0}, +> ] +> } +> } +> ``` ## Feature Properties -Feature properties are structured according to a **schema**. A schema has a set of **classes** and **enums**. A class contains a set of **properties**, which may be numeric, boolean, string, enum, or array types. - -A **feature** is a specific instantiation of class containing **property values**. Property values are stored in either a **property table** or a **property texture** depending on the use case. Both formats are designed for storing property values for a large number of features. - -By default, properties do not have any inherent meaning. A property may be assigned a **semantic**, an identifier that describes how the property should be interpreted. Built-in semantics include `ID` and `NAME`, as defined below. - -- `ID`: Unique identifier for the feature. -- `NAME`: Name of the feature; not required to be unique. +### Overview + +Feature properties describe attributes or characteristics of a feature. Schemas are templates describing the data types and semantic meanings of properties, where each feature is a single instance of that template with specific values. Property values may be associated with features in one of two ways: + +- **Property Tables** store property values as numeric arrays in a parallel, column-based binary layout. Property tables are indexed by Feature IDs, used as the index of a given feature into each property array. +- **Property Textures** store property values in channels of a texture, suitable for very high-frequency data mapped to less-detailed 3D surfaces. Property textures are indexed by texture coordinates, and do not have associated Feature IDs. + +Both storage formats are appropriate for efficiently transmitting large quantities of property values. + +### Schema Definitions + +#### Overview + +Data types and semantic meanings of properties are provided by a schema, as defined in the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/blob/3d-tiles-next/specification/Metadata/) and summarized below. + +#### Schema + +*Defined in [schema.schema.json](./schema/schema.schema.json).* + +Top-level definitions for type and semantic information. The schema provides a set of *classes* and *enums* the asset can reference. + +A schema may be embedded in the extension directly or referenced externally with the `schemaUri` property. Multiple glTF assets may refer to the same external schema to avoid duplication. A schema is defined by an `EXT_mesh_features` extension attached to the glTF root object. + +> **Example:** A simple schema defining enums and classes. +> +> ```jsonc +> { +> "extensions": { +> "EXT_mesh_features": { +> "schema": { +> "name": "Schema 001", +> "description": "An example schema.", +> "version": "3.5.1", +> "enums": { ... }, +> "classes": { ... } +> } +> } +> } +> } +> ``` + +#### Class + +*Defined in [class.schema.json](./schema/class.schema.json).* + +Template for features. Classes provide a list of properties with type and semantic information. Every feature must be associated with a class, and the feature's properties must conform to the class's property definitions. Features whose properties conform to a class are considered instances of that class. + +Classes are defined as entries in the `schema.classes` dictionary, indexed by an alphanumeric class ID. + +> **Example:** A "Tree" class, which might describe a table of tree measurements taken in a park. Property definitions abbreviated here, and introduced in the next section. +> +> ```jsonc +> { +> "extensions": { +> "EXT_mesh_features": { +> "schema": { +> "classes": { +> "tree": { +> "name": "Tree", +> "description": "Woody, perennial plant.", +> "properties": { +> "species": { ... }, +> "height": { ... }, +> "diameter": { ... } +> } +> } +> } +> } +> } +> } +> } +> ``` + +#### Class Property + +*Defined in [class.property.schema.json](./schema/class.property.schema.json).* + +Properties are defined abstractly in a class by their semantic meaning and data type, and are instantiated in a feature with specific values conforming to that definition. Properties support a richer variety of data types than glTF accessors or GPU shading languages allow, defined by `property.componentType`: + +- `componentType` + - **Type:** `string` + - **Required:** ✓ Yes + - **Allowed values:** + - `"INT8"` + - `"UINT8"` + - `"INT16"` + - `"UINT16"` + - `"INT32"` + - `"UINT32"` + - `"INT64"` + - `"UINT64"` + - `"FLOAT32"` + - `"FLOAT64"` + - `"BOOLEAN"` + - `"STRING"` + - `"ENUM"` + +A property may compose multiple components into higher-level types (vector, matrix, and array), defined by `property.type`: + +- `type` + - **Type:** `string` + - **Required:** No, default: `"SINGLE"` + - **Allowed values:** + - `"SINGLE"` + - `"VEC2"` + - `"VEC3"` + - `"VEC4"` + - `"MAT2"` + - `"MAT3"` + - `"MAT4"` + - `"ARRAY"` + +Class properties are defined as entries in the `class.properties` dictionary, indexed by an alphanumeric property ID. + +> **Example:** A "Tree" class, which might describe a table of tree measurements taken in a park. Properties include species, height, and diameter of each tree, as well as the number of birds observed in its branches. +> +> ```jsonc +> { +> "extensions": { +> "EXT_mesh_features": { +> "schema": { +> "classes": { +> "tree": { +> "name": "Tree", +> "description": "Woody, perennial plant.", +> "properties": { +> "species": { +> "description": "Type of tree.", +> "componentType": "ENUM", +> "enumType": "speciesEnum", +> "required": true +> }, +> "birdCount": { +> "description": "Number of birds perching on the tree", +> "type": "UINT8", +> "required": true +> }, +> "height": { +> "description": "Height of tree measured from ground level, in meters.", +> "componentType": "FLOAT32" +> }, +> "diameter": { +> "description": "Diameter at trunk base, in meters.", +> "componentType": "FLOAT32" +> } +> } +> } +> } +> } +> } +> } +> } +> ``` + +#### Enum + +*Defined in [enum.schema.json](./schema/enum.schema.json).* + +Set of categorical types, defined as `name: integer` pairs. Enum properties use an enum as their data type. + +Enums are defined as entries in the `schema.enums` dictionary, indexed by an alphanumeric enum ID. + +> **Example:** A "Color" enum, defining pixel colors on a simple RGB display. Use of a leading "Unspecified" enum value is optional, but suggested where missing data may be present. +> +> ```jsonc +> { +> "extensions": { +> "EXT_mesh_features": { +> "schema": { +> "enums": { +> "speciesEnum": { +> "name": "Species", +> "description": "An example enum for tree species.", +> "values": [ +> {"name": "Unspecified", "value": 0}, +> {"name": "Oak", "value": 1}, +> {"name": "Pine", "value": 2}, +> {"name": "Walnut", "value": 3} +> ] +> } +> } +> } +> } +> } +> } +> ``` + +#### Enum Values + +*Defined in [enum.value.schema.json](./schema/enum.value.schema.json).* + +Pairs of `name: integer` entries representing possible values of an enum property. + +Enum values are defined as entries in the `enum.values` array. Duplicate names or duplicate integer values are not allowed. -Model authors may define their own application- or domain-specific semantics separately. For application-specific semantics, authoring implementations are encouraged to use a `_*` prefix. For semantics common to a particular domain or vendor, creation of new uppercase, alphanumeric prefixes is encouraged. - -This extension implements the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata), which describes the metadata format in full detail. - -### Schemas +### Property Tables -A schema defines a set of classes and enums used in a model. Classes serve as templates for features - they provide a list of properties and the type information for those properties. Enums define the allowable values for enum properties. Schemas are defined in full detail in the [Schemas](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata#schemas) section of the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata). +*Defined in [propertyTable.schema.json](./schema/propertyTable.schema.json).* + +Each property table defines a specified number (`count`) of features conforming to a particular class (`class`), with property values stored as parallel arrays in a column-based binary layout. Property tables support a richer variety of data types than glTF accessors or GPU shading languages allow, and are suitable for datasets that can be expressed in a tabular layout. + +Property tables are defined as entries in the `propertyTables` array of the root-level `EXT_mesh_features` extension, and may be referenced by extensions on primitive or node objects. + +The property table may provide value arrays for only a subset of the properties of its class, but class properties marked `required: true` must not be omitted. Each property value array given by the property table must be defined by a class property with the same alphanumeric property ID, with values matching the data type of the class property. + +> **Example:** A `tree_survey_2021-09-29` property table, implementing the `tree` class defined in earlier examples. The table contains observations for 10 trees, with each property defined by a buffer view containing 10 values. +> +> ```jsonc +> { +> "extensions": { +> "EXT_mesh_features": { +> "schema": { ... }, +> "propertyTables": [{ +> "name": "tree_survey_2021-09-29", +> "class": "tree", +> "count": 10, +> "properties": { +> "species": { +> "bufferView": 2, +> "stringOffsetBufferView": 3 +> }, +> "birdCount": { +> "bufferView": 1 +> }, +> "height": { +> "bufferView": 0 +> }, +> // "diameter" is not required and has been omitted from this table. +> } +> }] +> } +> } +> } +> ``` + +Property arrays are stored in glTF buffer views and use the binary encoding defined in the [Table Format](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata#table-format) section of the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata). -A schema may be embedded in the extension directly or referenced externally with the `schemaUri` property. Multiple glTF models may refer to the same external schema to avoid duplication. +As in the core glTF specification, values of NaN, +Infinity, and -Infinity are never allowed. -Schemas may be given a `name`, `description`, and `version`. +Each buffer view `byteOffset` must be aligned to a multiple of its component size. -### Property Tables +> **Implementation note:** Authoring tools may choose to align all buffer views to 8-byte boundaries for consistency, but client implementations should only depend on 8-byte alignment for buffer views containing 64-bit component types. -A property table stores property values in a parallel array format. Each property array corresponds to a class property. The values contained within a property array must match the data type of the class property. Furthermore, the set of property arrays must match one-to-one with the class properties. There is one exception - if a property specifies a `noData` value, the property table may omit that property. - -The schema and property tables are defined in the root extension object in the glTF model. See the example below: - -```jsonc -{ - "extensions": { - "EXT_mesh_features": { - "schema": { - "classes": { - "tree": { - "properties": { - "height": { - "description": "Height of tree measured from ground level", - "type": "FLOAT32" - }, - "birdCount": { - "description": "Number of birds perching on the tree", - "type": "UINT8" - }, - "species": { - "description": "Species of the tree", - "type": "STRING" - } - } - } - } - }, - "propertyTables": [{ - "name": "tree", - "class": "tree", - "count": 10, - "properties": { - "height": { - "bufferView": 0 - }, - "birdCount": { - "bufferView": 1 - }, - "species": { - "bufferView": 2, - "stringOffsetBufferView": 3 - } - } - }] - } - } -} -``` - -`class` is the ID of the class in the schema. `count` is the number of features in the property table, as well as the length of each property array. Property arrays are stored in glTF buffer views and use the binary encoding defined in the [Table Format](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata#table-format) section of the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata). +### Property Textures -As in the core glTF specification, values of NaN, +Infinity, and -Infinity are never allowed. +*Defined in [propertyTexture.schema.json](./schema/propertyTexture.schema.json).* -Each buffer view `byteOffset` must be aligned to a multiple of 8 bytes. +Property textures use texture channels to store property values conforming to a particular class (identified by ID `class`), with those values accessed directly by texture coordinates. Property textures do not require feature IDs, and are especially useful when texture mapping high frequency data to less detailed 3D surfaces. Unlike textures used in glTF materials, property textures are not necessarily visible in a rendered scene. Like property tables, property textures are implementations of the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata). -### Property Textures +Property textures are defined as entries in the `propertyTextures` array of the root-level `EXT_mesh_features` extension, and may be referenced by extensions on primitive objects. Property textures do not provide per-instance values with `EXT_mesh_gpu_instancing`, and must not be used by extensions on node objects. -Property textures use textures rather than parallel arrays to store values. Property textures are accessed directly by texture coordinates, and do not require feature IDs. Property textures are especially useful when texture mapping high frequency data to less detailed 3D surfaces. For each property that does not specify a `noData` value, a mapping to the corresponding texture channel or channels is required. Properties with a `noData` value are optional in property textures instantiating a given class. +A property texture may provide channels for only a subset of the properties of its class, but class properties marked `required: true` must not be omitted. -Property textures use the [Raster Format](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata#raster-format) of the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata) with a few additional constraints: +Several constraints are imposed to ensure compatibility of texture storage with various property types: * A scalar property cannot be encoded into multiple channels. For example, it is not possible to encode a `UINT32` property in an `RGBA8` texture. -* Components of fixed-length array properties must be separate channels within the same texture. +* Components of array and vector properties must be separate channels within a texture. * Variable-length arrays are not supported. - -Additionally, the data type and bit depth of the image must be compatible with the property type. An 8-bit per pixel RGB image is only compatible with `UINT8` or normalized `UINT8` properties, and array properties thereof with three components or less. Likewise, a floating point property requires a floating point-compatible image format like KTX2 which may require additional extensions. - -Property textures are defined with the following steps: - -1. A class is defined in the root `EXT_mesh_features` extension object. This is used to describe the properties in the texture. -2. A property texture is defined in the root `EXT_mesh_features.propertyTextures` object. This must reference the class ID defined in step 1. -3. A property texture is associated with a primitive by listing the property texture ID in the `primitive.EXT_mesh_features.propertyTextures` array. -Property Texture - -_Class and property texture_ - -```jsonc -{ - "extensions": { - "EXT_mesh_features": { - "schema": { - "classes": { - "heatSample": { - "properties": { - "heatLoss": { - "type": "UINT8", - "normalized": true - }, - "insulation": { - "type": "UINT8", - "normalized": true - }, - } - } - } - }, - "propertyTextures": [{ - "class": "heatSample", - "index": 0, - "texCoord": 0, - "properties": { - "heatLoss": [0], - "insulation": [1] - } - }] - } - } -} -``` - -_Primitive_ - -```jsonc -{ - "primitives": [ - { - "attributes": { - "POSITION": 0, - "TEXCOORD_0": 1 - }, - "indices": 2, - "material": 0, - "extensions": { - "EXT_mesh_features": { - "propertyTextures": [0] - } - } - } - ] -} -``` - - -A `propertyTexture` object extends the glTF [`textureInfo`](../../../../../specification/2.0/schema/textureInfo.schema.json) object. `texCoord` refers to the texture coordinate set of the referring primitive. The `properties` map specifies the texture channels providing data for all required class properties, and perhaps optional class properties. An array of integer index values identify channels, where multiple channels may be used only for fixed-length arrays of 2, 3, or 4 components. Channels of an `RGBA` texture are numbered 0–3 respectively, although specialized texture formats may allow additional channels. All values are stored in linear space. +* Data type and bit depth of the image must be compatible with the property type. + +For example, an 8-bit per pixel RGB image may only contain `SINGLE`, fixed-length `ARRAY` (length ≤3), `VEC2`, or `VEC3` values composed of 8-bit component types. + +Enum values may be encoded in images, as integer values according to their enum value type (see [Enum](#enum)). + +> **Implementation note:** Use of floating-point properties in a property texture would require a floating point-compatible image format like KTX2, provided by an additional extension. + +> **Example:** Property texture implementing a "wall" class, with property values stored in a glTF texture at index 0 and indexed by `TEXCOORD_0`. +> +> Property Texture +> +> _Class and property texture_ +> +> ```jsonc +> { +> "extensions": { +> "EXT_mesh_features": { +> "schema": { +> "classes": { +> "wall": { +> "name": "Wall Temperature vs. Insulation", +> "properties": { +> "insideTemp": { +> "name": "Inside Temperature", +> "type": "UINT8" +> }, +> "outsideTemp": { +> "name": "Outside Temperature", +> "type": "UINT8" +> }, +> "insulation": { +> "name": "Insulation Thickness", +> "type": "UINT8", +> "normalized": true +> }, +> } +> } +> } +> }, +> "propertyTextures": [{ +> "class": "wall", +> "index": 0, +> "texCoord": 0, +> "properties": { +> "insideTemp": [0], +> "outsideTemp": [1], +> "insulation": [2] +> } +> }] +> } +> } +> } +> ``` +> +> _Primitive_ +> +> ```jsonc +> { +> "primitives": [ +> { +> "attributes": { +> "POSITION": 0, +> "TEXCOORD_0": 1 +> }, +> "indices": 2, +> "material": 0, +> "extensions": { +> "EXT_mesh_features": { +> "propertyTextures": [0] +> } +> } +> } +> ] +> } +> ``` + + +A `propertyTexture` object extends the glTF [`textureInfo`](../../../../../specification/2.0/schema/textureInfo.schema.json) object. `texCoord` specifies a texture coordinate set in the referring primitive. + +The `properties` map specifies the texture channels providing data for available properties. An array of integer index values identifies channels, where multiple channels may be used only for fixed-length arrays of 2, 3, or 4 components. Channels of an `RGBA` texture are numbered 0–3 respectively, although specialized texture formats may allow additional channels. All values are stored in linear space. + +> **Example:** A property texture for wind velocity samples. The "speedKPH" property values are stored in the red channel, and "direction" property values are stored as a unit-length vector, with X/Y components in the green and blue channels. Both properties are indexed by UV coordinates in a `TEXCOORD_0` attribute. +> +> ```jsonc +> // Root EXT_mesh_features extension: +> { +> "propertyTextures": [{ +> "class": "wind", +> "index": 0, +> "texCoord": 0, +> "properties": { +> "speedKPH": [0], +> "direction": [1, 2] +> } +> }] +> } Texture filtering must be `9728` (NEAREST), `9729` (LINEAR), or undefined, for any texture object referenced as a property texture. ## Binary Data Storage -`EXT_mesh_features` imposes additional binary data alignment requirements on an asset, extending the 4-byte alignment in the core glTF specification: +Feature properties are stored in a compact binary tabular format described in the [Cesium 3D Metadata Specification](https://github.com/CesiumGS/3d-tiles/tree/3d-tiles-next/specification/Metadata), with each property table array occupying a glTF buffer view. `EXT_mesh_features` imposes 8-byte binary data alignment requirements on an asset, allowing support for 64-bit data types while remaining compatible with the 4-byte alignments in the core glTF specification: - GLB-stored `JSON` chunk must be padded with trailing `Space` characters (`0x20`) to 8-byte boundary. - GLB-stored `BIN` chunk must be padded with trailing zeroes (`0x00`) to 8-byte boundary. As a result, byte length of the `BIN` chunk may be up to 7 bytes larger than JSON-defined `buffer.byteLength` to satisfy alignment requirements. +## Optional vs. Required + +This extension is optional, meaning it should be placed in the `extensionsUsed` list, but not in the `extensionsRequired` list. + +## Schema + +* [gltf.EXT_mesh_features.schema.json](./schema/gltf.EXT_mesh_features.schema.json) +* [primitive.EXT_mesh_features.schema.json](./schema/primitive.EXT_mesh_features.schema.json) +* [node.EXT_mesh_features.schema.json](./schema/node.EXT_mesh_features.schema.json) + ## Examples _This section is non-normative_ @@ -393,20 +653,14 @@ The examples below shows the breadth of possible use cases for this extension. Example|Description|Image --|--|-- Triangle mesh|Feature IDs are assigned to each vertex to distinguish components of a building.|![Building Components](figures/building-components.png) -Per-vertex metadata|An implicit feature ID is assigned to each vertex. The property table stores `FLOAT64` accuracy values. |![Per-vertex metadata](figures/per-vertex-metadata.png) -Per-triangle metadata|An implicit feature ID is assigned to each set of three vertices. The property table stores `FLOAT64` area values.|![Per-triangle metadata](figures/per-triangle-metadata.png) -Per-point metadata|An implicit feature ID is assigned to each point. The property table stores `FLOAT64` , `STRING`, and `ENUM` properties, which are not possible through glTF vertex attribute accessors alone.|![Point features](figures/point-features.png) -Per-node metadata|Vertices in node 0 and node 1, not batched together, are assigned different `offset` feature IDs.|![Per-node metadata](figures/per-node-metadata.png) -Multi-point features|A point cloud with two property tables, one storing metadata for groups of points and the other storing metadata for individual points.|![Multi-point features](figures/point-cloud-layers.png) -Multi-instance features|Instanced tree models where trees are assigned to groups with a per-instance feature ID attribute. One property table stores per-group metadata and the other stores per-tree metadata.|![Multi-instance features](figures/multi-instance-metadata.png) -Material classification|A textured mesh using a property texture to store both material enums and normalized `UINT8` thermal temperatures.|![Material Classification](figures/material-classification.png) -Composite|A glTF containing a 3D mesh (house), a point cloud (tree), and instanced models (fencing) with three property tables.|![Composite Example](figures/composite-example.png) - -## Schema - -* [gltf.EXT_mesh_features.schema.json](./schema/gltf.EXT_mesh_features.schema.json) -* [primitive.EXT_mesh_features.schema.json](./schema/primitive.EXT_mesh_features.schema.json) -* [node.EXT_mesh_features.schema.json](./schema/node.EXT_mesh_features.schema.json) +Per-vertex properties|An implicit feature ID is assigned to each vertex. The property table stores `FLOAT64` accuracy values. |![Per-vertex properties](figures/per-vertex-metadata.png) +Per-triangle properties|An implicit feature ID is assigned to each set of three vertices. The property table stores `FLOAT64` area values.|![Per-triangle properties](figures/per-triangle-metadata.png) +Per-point properties|An implicit feature ID is assigned to each point. The property table stores `FLOAT64` , `STRING`, and `ENUM` properties, which are not possible through glTF vertex attribute accessors alone.|![Point features](figures/point-features.png) +Per-node properties|Vertices in node 0 and node 1, not batched together, are assigned different `offset` feature IDs.|![Per-node properties](figures/per-node-metadata.png) +Multi-point features|A point cloud with two property tables, one storing properties for groups of points and the other storing properties for individual points.|![Multi-point features](figures/point-cloud-layers.png) +Multi-instance features|Instanced tree meshes, where trees are assigned to groups with a per-GPU-instance feature ID attribute. One property table stores per-group properties and the other stores per-tree properties.|![Multi-instance features](figures/multi-instance-metadata.png) +Material classification|A textured mesh using a property texture to store both material enums and normalized `UINT8` insulation values.|![Material Classification](figures/material-classification.png) +Composite|A glTF containing a 3D mesh (house), a point cloud (tree), and instanced meshes (fencing) with three property tables.|![Composite Example](figures/composite-example.png) ## Revision History @@ -441,3 +695,5 @@ Composite|A glTF containing a 3D mesh (house), a point cloud (tree), and instanc * Renamed `featureTable` → `propertyTable` and `featureTexture` → `propertyTexture` * Removed `featureIdAttributes` and `featureIdTextures`, replaced with `featureIds` * Removed string ID references to property tables and textures, replaced with integer IDs + * Relaxed buffer view alignment to component size, rather than strict 8-byte boundaries + * Clarified that nodes with GPU instancing cannot reference property textures diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-id-texture.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-id-texture.png index 09610050df..d1e0a19f10 100644 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-id-texture.png and b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-id-texture.png differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table-buildings.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table-buildings.png deleted file mode 100644 index ee170d623e..0000000000 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table-buildings.png and /dev/null differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table.png deleted file mode 100644 index ffc3cefb7d..0000000000 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table.png and /dev/null differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table.svg b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table.svg new file mode 100644 index 0000000000..e1b79a519f --- /dev/null +++ b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-table.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-texture.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-texture.png index e1a3f2d81a..83687befbe 100644 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-texture.png and b/extensions/2.0/Vendor/EXT_mesh_features/figures/feature-texture.png differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/material-classification.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/material-classification.png index 7b0e150d8e..9679e7c7ac 100644 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/material-classification.png and b/extensions/2.0/Vendor/EXT_mesh_features/figures/material-classification.png differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/metadata-access.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/metadata-access.png deleted file mode 100644 index b7c043cb16..0000000000 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/metadata-access.png and /dev/null differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/placemarks.jpg b/extensions/2.0/Vendor/EXT_mesh_features/figures/placemarks.jpg new file mode 100644 index 0000000000..c2f93f96b7 Binary files /dev/null and b/extensions/2.0/Vendor/EXT_mesh_features/figures/placemarks.jpg differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/figures/placemarks.png b/extensions/2.0/Vendor/EXT_mesh_features/figures/placemarks.png deleted file mode 100644 index eb39992a49..0000000000 Binary files a/extensions/2.0/Vendor/EXT_mesh_features/figures/placemarks.png and /dev/null differ diff --git a/extensions/2.0/Vendor/EXT_mesh_features/schema/class.property.schema.json b/extensions/2.0/Vendor/EXT_mesh_features/schema/class.property.schema.json index eb81480f66..0be3f486f5 100644 --- a/extensions/2.0/Vendor/EXT_mesh_features/schema/class.property.schema.json +++ b/extensions/2.0/Vendor/EXT_mesh_features/schema/class.property.schema.json @@ -58,7 +58,7 @@ }, "normalized": { "type": "boolean", - "description": "Specifies whether integer values are normalized. This applies both when `type` is an integer type, or when `type` is `ARRAY` with a `componentType` that is an integer type. For unsigned integer types, values are normalized between `[0.0, 1.0]`. For signed integer types, values are normalized between `[-1.0, 1.0]`. For all other types, this property is ignored.", + "description": "Specifies whether integer values are normalized. This applies both when `componentType` is an integer type. For unsigned integer component types, values are normalized between `[0.0, 1.0]`. For signed integer component types, values are normalized between `[-1.0, 1.0]`. For all other component types, this property must be false.", "default": false }, "max": { diff --git a/extensions/2.0/Vendor/EXT_mesh_features/schema/schema.schema.json b/extensions/2.0/Vendor/EXT_mesh_features/schema/schema.schema.json index 35df7085ec..c0e7795b2c 100644 --- a/extensions/2.0/Vendor/EXT_mesh_features/schema/schema.schema.json +++ b/extensions/2.0/Vendor/EXT_mesh_features/schema/schema.schema.json @@ -7,7 +7,7 @@ "name": { "type": "string", "minLength": 1, - "description": "The name of the schema." + "description": "The name of the schema, e.g. for display purposes." }, "description": { "type": "string",