Fixes to quantized sequence and scramble decode #987

keptsecret · 2026-01-09T08:51:58Z

No description provided.

…mconstant in quantized sequence

…o order stored values from LSB to MSB

devshgraphicsprogramming · 2026-01-15T16:38:42Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+template<typename T>
+NBL_BOOL_CONCEPT SequenceSpecialization = concepts::UnsignedIntegral<typename vector_traits<T>::scalar_type> && size_of_v<typename vector_traits<T>::scalar_type> <= 4;


could have just used concepts::IntVector<T> instead of the UnsignedIntegral

devshgraphicsprogramming · 2026-01-15T16:41:09Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+vector<R,D> decode(NBL_CONST_REF_ARG(QuantizedSequence<T, D>) val, const vector<unsigned_integer_of_size_t<sizeof(R)>,D> scrambleKey)
 {
-    return impl::decode_helper<T,D,EncodeScramble>::__call(val, scrambleKey);
+    return impl::decode_before_scramble_helper<T,D>::__call(val, scrambleKey);
 }

-#define SEQUENCE_SPECIALIZATION_CONCEPT concepts::UnsignedIntegral<typename vector_traits<T>::scalar_type> && size_of_v<typename vector_traits<T>::scalar_type> <= 4
+// pre-decode scramble
+template<typename R, typename T, uint16_t D>
+vector<R,D> decode(NBL_CONST_REF_ARG(QuantizedSequence<T, D>) val, NBL_CONST_REF_ARG(QuantizedSequence<T, D>) scrambleKey)
+{
+    return impl::decode_after_scramble_helper<T,D>::__call(val, scrambleKey);
+}


I don't like these being "freely floating" as-in just sampling::decode they should be a NBL_CONST_METHOD of the QuantizedSequence

as you can see when its a free floating function there's an implicit const this as first parameter, so you can see that it wworks better conceptually as a method

then they'd only be template<typename F> vector<F,D> decode(scramblekey)

devshgraphicsprogramming · 2026-01-15T16:43:02Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

 template<typename T, uint16_t D>
-struct decode_helper<T, D, false>
+struct decode_before_scramble_helper
 {
    using scalar_type = typename vector_traits<T>::scalar_type;
-    using fp_type = typename float_of_size<sizeof(scalar_type)>::type;
-    using uvec_type = vector<scalar_type, D>;
+    using uvec_type = vector<uint32_t, D>;
    using sequence_type = QuantizedSequence<T, D>;
-    using return_type = vector<fp_type, D>;
-    NBL_CONSTEXPR_STATIC_INLINE scalar_type UNormConstant = unorm_constant<8u*sizeof(scalar_type)>::value;
+    using return_type = vector<float32_t, D>;
+    NBL_CONSTEXPR_STATIC_INLINE uint32_t UNormConstant = unorm_constant<float,8u*sizeof(scalar_type)>::value;

    static return_type __call(NBL_CONST_REF_ARG(sequence_type) val, const uvec_type scrambleKey)
    {
        uvec_type seqVal;
        NBL_UNROLL for(uint16_t i = 0; i < D; i++)
-            seqVal[i] = val.get(i) ^ scrambleKey[i];
-        return return_type(seqVal) * bit_cast<fp_type>(UNormConstant);
+            seqVal[i] = val.get(i);
+        seqVal ^= scrambleKey;
+        return return_type(seqVal) * bit_cast<float_of_size_t<sizeof(scalar_type)> >(UNormConstant);
    }
 };
 template<typename T, uint16_t D>
-struct decode_helper<T, D, true>


its confusing and digusting that they're templated on T,D which are storage type and dimensions, it would be better to template them on just the QuantizedSequence and return type' scalar type F

Also the scalar_type alias here is a complete misnomer

devshgraphicsprogramming · 2026-01-15T17:03:36Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+template<typename T> NBL_PARTIAL_REQ_TOP(impl::SequenceSpecialization<T>)
+struct QuantizedSequence<T, 1 NBL_PARTIAL_REQ_BOT(impl::SequenceSpecialization<T>) >
 {
    using store_type = T;
-    NBL_CONSTEXPR_STATIC_INLINE uint32_t UNormConstant = impl::unorm_constant<8u*sizeof(store_type)>::value;
+    NBL_CONSTEXPR_STATIC_INLINE uint16_t BitsPerComponent = 8u*size_of_v<store_type>;

    store_type get(const uint16_t idx) { assert(idx > 0 && idx < 1); return data; }


make all QuantizedSequence have an create for consistency

Also have a

template<typename unorm_t> requires (sizeof(unorm_t)*8>=BitsPerComponent) encode(const vector<unorm_t,D>)

which assumes you're feeding full range sizeof(unorm_t)*8-bit random values and pre-shifts right by sizeof(unorm_t)*8-BitsPerComponent (to remove extra LSB bits) before calling create so this treatment is not buried somewhere in your C++ code which fills the sequence

devshgraphicsprogramming · 2026-01-15T17:05:13Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+        QuantizedSequence<T, Dim> seq;
+        NBL_UNROLL for (uint16_t i = 0; i < Dim; i++)


you forgot to set seq.data = 0 first, you have UB (compiler will OpUndef assign and then any expression involving this will get optimized away - source of bugs)

devshgraphicsprogramming · 2026-01-15T17:06:08Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+        QuantizedSequence<T, Dim> seq;
+        NBL_UNROLL for (uint16_t i = 0; i < Dim; i++)
+            seq.set(i, value[i]);


same stuff about intializing to 0

devshgraphicsprogramming · 2026-01-15T17:06:21Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+        QuantizedSequence<T, Dim> seq;
+        NBL_UNROLL for (uint16_t i = 0; i < Dim; i++)
+            seq.set(i, value[i]);
+        return seq;


again, initialize to 0

devshgraphicsprogramming · 2026-01-15T17:07:49Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+// uint32_t4; Dim=3 --> returns uint32_t2 - 42 bits per component: 32 in x, 10 in y
+// use uint32_t2 instead of uint16_t4
+template<typename T, uint16_t Dim> NBL_PARTIAL_REQ_TOP(impl::SequenceSpecialization<T> && size_of_v<typename vector_traits<T>::scalar_type> == 4 && vector_traits<T>::Dimension == 4 && Dim == 3)
+struct QuantizedSequence<T, Dim NBL_PARTIAL_REQ_BOT(impl::SequenceSpecialization<T> && size_of_v<typename vector_traits<T>::scalar_type> == 4 && vector_traits<T>::Dimension == 4 && Dim == 3) >


remove this for now, because it would require a decode of the sequence into float64_t to use for anything

devshgraphicsprogramming · 2026-01-15T17:10:55Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

-struct QuantizedSequence<T, Dim NBL_PARTIAL_REQ_BOT(SEQUENCE_SPECIALIZATION_CONCEPT && vector_traits<T>::Dimension == 2 && Dim == 4) >
+// uint16_t2; Dim=4 -- should use uint16_t4 instead of uint32_t2
+template<typename T, uint16_t Dim> NBL_PARTIAL_REQ_TOP(impl::SequenceSpecialization<T> && size_of_v<typename vector_traits<T>::scalar_type> == 2 && vector_traits<T>::Dimension == 2 && Dim == 4)
+struct QuantizedSequence<T, Dim NBL_PARTIAL_REQ_BOT(impl::SequenceSpecialization<T> && size_of_v<typename vector_traits<T>::scalar_type> == 2 && vector_traits<T>::Dimension == 2 && Dim == 4) >


the specializations only meant to match uint16_t2 should just check with is_same_v<T,uint16_t2>

devshgraphicsprogramming · 2026-01-15T17:11:25Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+// uint32_t2; Dim=3 -- should never use uint16_t2 instead of uint32_t
+template<typename T, uint16_t Dim> NBL_PARTIAL_REQ_TOP(impl::SequenceSpecialization<T> && size_of_v<typename vector_traits<T>::scalar_type> == 4 && vector_traits<T>::Dimension == 2 && Dim == 3)
+struct QuantizedSequence<T, Dim NBL_PARTIAL_REQ_BOT(impl::SequenceSpecialization<T> && size_of_v<typename vector_traits<T>::scalar_type> == 4 && vector_traits<T>::Dimension == 2 && Dim == 3) >


Keep it Super Simple with is_same_v<T,uint32_t2> && Dim==3

devshgraphicsprogramming · 2026-01-15T17:12:25Z

include/nbl/builtin/hlsl/sampling/quantized_sequence.hlsl

+        if (idx >= 0 && idx < 2) // x y
+        {
+            return glsl::bitfieldExtract(data[0], BitsPerComponent * idx, BitsPerComponent);
+        }
+        else    // z w
+        {
+            return glsl::bitfieldExtract(data[1], BitsPerComponent * (idx & uint16_t(1u)), BitsPerComponent);
+        }
    }

    void set(const uint16_t idx, const scalar_type value)
    {
        assert(idx >= 0 && idx < 4);
-        const uint16_t i = (idx & uint16_t(2u)) >> uint16_t(1u);
-        const uint16_t odd = idx & uint16_t(1u);
-        data[i] &= hlsl::mix(~Mask, Mask, bool(odd));
-        data[i] |= ((value >> DiscardBits) & Mask) << (BitsPerComponent * odd);
+        if (idx >= 0 && idx < 2) // x y


you're asserting idx>=0 you don't need to check it in the if-statement

keptsecret added 4 commits January 9, 2026 10:25

changed unormconstant template params to include float

df94ffe

separate pre and post decode scramble into different structs, no unor…

2c1a99d

…mconstant in quantized sequence

change quantized sequence to set/get with bitfieldInsert/Extract, als…

8894dd1

…o order stored values from LSB to MSB

added create functions that should not preserve existing bits

639f464

devshgraphicsprogramming reviewed Jan 15, 2026

View reviewed changes

		template<typename T>
		NBL_BOOL_CONCEPT SequenceSpecialization = concepts::UnsignedIntegral<typename vector_traits<T>::scalar_type> && size_of_v<typename vector_traits<T>::scalar_type> <= 4;

		QuantizedSequence<T, Dim> seq;
		NBL_UNROLL for (uint16_t i = 0; i < Dim; i++)

Fixes to quantized sequence and scramble decode #987

Are you sure you want to change the base?

Fixes to quantized sequence and scramble decode #987

Uh oh!

Conversation

keptsecret commented Jan 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devshgraphicsprogramming Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devshgraphicsprogramming Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devshgraphicsprogramming Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

devshgraphicsprogramming Jan 15, 2026 •

edited

Loading

devshgraphicsprogramming Jan 15, 2026 •

edited

Loading

devshgraphicsprogramming Jan 15, 2026 •

edited

Loading