Integer vector SIMD support

[robclu]

Hi,

Thanks for the awesome work!

I have a use case which makes heavy use of integer vectors, which lead me to notice that there are no UVec3A and IVec3A types, and that the UVec4 and IVec4 types do not actually have SIMD implementations. Sorry if I have missed something and that there are in fact SIMD implementations for them, but if not, do you have plans to add them?

If not, I will work on a PR to add them if you'd want those implementations in the library.

[bitshifter]

Hi there. You are correct, there are no integer SIMD implementations at the moment.

I am in two minds about adding support. For one, it is quite a lot of work - especially to support all of the architectures that glam currently supports (although this isn't a blocker, missing implements have a scalar path). Another thing I have found when adding support for NEON is the benefits have been less clear than they were with SSE2. I am considering changing how glam supports SIMD - mostly just making it more opt in than it currently is for Vec4/Quat and Mat4, that wouldn't negate any integer SIMD work though, it would just mean that it moves. In any case if you are interested in doing the work then I would support a PR, but as I don't have a huge amount of time to spend on this myself it would need to be fairly complete.

[djeedai]

Hi-

I was wondering it there was any update on this? I absolutely appreciate the amount of work this is, for doing some maintenance myself on a similar library (not in Rust), so I'd understand if there's some time concern. But I'm also a bit perplex about the argument that there's little benefit to performance; I've been using SIMD quite a bit to optimize code for a decade in games and if you have some intensive math algorithm, in my experience it does make a difference. I know floating point math is a lot more ubiquitous, but there's plenty of applications of integer vector math when using grid-based data structures. And I appreciate that compilers do indeed auto-vectorize, but my experience is that this is pretty limited, and some more complex operations need to be written by hand.

Would it be acceptable to start implementing (that is accepting PRs; I think @robclu was interested too) under an "experimental" feature flag, to allow incrementally adding features without having to support the entire range of functions, benchmarks, tests, etc. that would come with a "proper" implementation like the float one?

[bitshifter]

Regarding the performance comment, I think when I made that there were some suboptimal parts of my neon implementation (from memory I'd missed converting a few things). There was some good performance benefits seen in some workloads once it was done, bevy's occlusion culling for example saw a large benefit in their stress test.

I think if people felt there were real benefits to this then I'd certainly consider it. I think it's simultaneously simpler in some ways (no quat, matrix or affine) and more complex (there are a lot of integer types). The float implementation was really driven by benchmarking which showed benefits (see https://bitshifter.github.io/2018/06/04/simd-path-tracing/#converting-vec3-to-sse2) and I think if glam took this path that should be the case for integer simd as well, it's a lot of effort and it will introduce additional complexity both in the API and the implementation so the benefits need to be demonstrable.

So I'm definitely open to it, I'd probably prefer a branch to an experimental feature flag at least to start with. I don't necessarily have time to do it myself at the moment but more importantly I don't have a strong use case and I think this kind of thing works better when it's used in anger.

Some questions that would be good to answer:

• Should there be an IVec3A & UVec3A?
• Which types does it make sense for, just 3D and 4D i32 and u32 types?
• Are there benchmarks that show these are faster (doesn't need to be a glam implementation, a POC would be OK)?

[djeedai]

The performance-driven approach makes sense. The branch also (I thought about it just after I sent my comment).

On the organization and the many integral types, I think there's a debate between the all-typed approach you find in e.g. the C++ SSE and NEON headers, and the generic integral register approach. The former is supposed to prevent typo errors if you use the wrong instruction by mistake. But in practice I've seen several devs complain that the NEON API in particular was annoying because you need to reinterpret-cast all the time, and those casts are no-op at the hardware level, but are very verbose in code. From my (limited) experience, integral vector usage tend to be more prone to back and forth between different integer types (that is, mixing instructions with different lane sizes), much more so than floating ones. This is due to the fact taking the upper N bits of a integer form another valid integer (e.g. top 16 bits of a u32 form a valid u16), unlike floating points where the IEEE encoding is all or nothing. Another example, you store u8 data, but use u32 instructions to manipulate 4 values as a block (especially with shifts, when the values in the SIMD register represents 2D or 3D data and you want to move rows or columns). So my advice would be to start with a general integral vector type IntVec3A / IntVec4 (as opposed to I for s32 or U for u32) which allows all operations for any lane size, and then maybe have some trait IntVec4x32 that restricts the operations to one lane size, if the user really wants to enforce a particular semantic in a given context. I'd even ignore IntVec3A entirely; the maintenance cost probably doesn't justify having a separate type instead of just using IntVec4 and ignoring the last component. And often if you optimize your code you'll find something useful to fit into that last component anyway (e.g. a 3D point + a scalar distance).

Having only i32 and u32 would be a start but as mentioned many algorithms use several lane sizes so that'd be restricting. I unfortunately don't know of a benchmark on top of my head.

[bitshifter]

Which types does it make sense for, just 3D and 4D i32 and u32 types?

By this I meant IVec3A, IVec4, UVec3A & UVec4, in case I wasn't clear.

I'd even ignore IntVec3A entirely; the maintenance cost probably doesn't justify having a separate type instead of just using IntVec4 and ignoring the last component. And often if you optimize your code you'll find something useful to fit into that last component anyway (e.g. a 3D point + a scalar distance).

Sure, if that would work for people it simplifies things

I unfortunately don't know of a benchmark on top of my head.

I don't mean an existing benchmark, more if there's some algorithm that you think this would speed up, it could be added to benches and then tested with or without SIMD. With could be a proof of concept before doing all the work making the tera templates generate SIMD types.