feat(stores): Replace Map backend with O(1) sparse set

[jtnuttall]

Hey folks,

This PR refactors the Map store's backend. It swaps out the existing IntMap-based implementation for one using sparse sets.

The tl;dr is that this seems to offer a really nice performance boost, making the default Map as fast as the old Cache(IntMap) setup out-of-the-box, with even better performance for Unboxed and Storable components via new specialized map types.

I was curious how this sparse-set approach would perform, and the initial numbers looked promising enough that I thought it was worth sharing. I know it's a big architectural change, so I'm keen to get feedback on the trade-offs before taking it any further.

The Motivation

I've been hacking on a small rendering engine in Haskell in my spare time and wrote heph-sparse-set as part of that project. I was happy with how it turned out and ended up publishing it as its own little library.

Since I was already using apecs as a testbed while developing it, and the benchmark numbers looked pretty good, I thought I'd clean up the integration work and submit it for real.

New Features & API Changes

• New Stores: Adds UMap and SMap for components that have Unbox or Storable instances.
• Pre-allocation: The new map types also come in MapWith, UMapWith, and SMapWith flavors. These take a type-level Nat to hint at the initial capacity, which helps avoid reallocations in hot loops.
• Cache Deprecation: Cache doesn't seem to provide any real benefit with the new stores. I've left the Cachable instance for backward compatibility, but I've marked it as obsolete in the docs.

Breaking Changes / Downsides

• Minimum GHC is now 8.10.
• The vector lower bound is now 0.12.3.0.
• heph-sparse-set was published yesterday, so it's not in any stackage snapshots.
• Map is now a type synonym around MapWith 16, so its semantics change quite a bit. This could be fixed with a newtype wrapper, but that seemed overengineered.

Performance Results

To get a clearer picture, I updated the benchmark suite to be a bit more rigorous (100k entities, 1k simulation steps). Of course, real-world performance will vary, but the results from the benchmark suite are below.

1. Comparison Against the Old IntMap Baseline

The benchmarks suggest a significant speedup compared to the original default Map.

Store Strategy	step Time (per step)	Speedup vs. Old Map
Old IntMap (Baseline)	2,299 µs	--
Old Cache(IntMap)	91.5 µs	25.0x
New Map (Sparse Set)	83.7 µs	~27x
New UMap (Unbox)	59.0 µs	~39x
New SMap (Storable)	51.0 µs	~45x

2. Comparison Against the Old Cache(IntMap) Baseline

The new default Map appears to match the old cached performance without any particular capacity tuning. There are further gains from the specialized maps. The difference is likely to be even more stark if we were to look at cases where we overflow the cache.

Store Strategy	step Time (per step)	Speedup vs. Old Cache(Map)
Old Cache(IntMap)	91.5 µs	--
New Map (Sparse Set)	83.7 µs	~1.0x
New UMap (Unbox)	59.0 µs	~1.5x
New SMap (Storable)	51.0 µs	~2x

Raw Criterion reports are on my fork if you want to dig in: before and after.

Discussion Points

This change is definitely a trade-off: it prioritizes CPU performance at the cost of memory efficiency in some cases.

• MapWith capacity: Right now, the implementation uses n as the dense capacity and n * 2 as the sparse capacity to reduce API complexity. This is a very fuzzy heuristic, and probably an incorrect one in most cases. Providing both parameters as Nats is the superior choice here. heph-sparse-set maintains its own internal invariants around minimum capacity, so an absurd capacity (0 or capacity(dense) > capacity(sparse)) shouldn't present issues beyond degraded performance.
• Memory Footprint: A sparse set's memory usage scales with the highest entity ID, not just the component count. Given that apecs uses a simple, monotonically increasing EntityCounter, this could lead to higher memory usage in long-running apps with a lot of entity churn.
• Compaction: The heph-sparse-set library has a compact operation that can mitigate this memory growth. The open question for this PR is how we should handle that. apecs doesn't currently have a mechanism to trigger compaction, and its ability to constrain resources would be limited without a mechanism to constrain entity ID growth over time, so I'm interested in hearing any thoughts on what a good API for that might look like.

[jonascarpay]

Thanks for an exemplary PR. I'm a little pressed for time right now, but I want to leave some initial thoughts.

• I like this a lot. A general-purpose "default" map is very nice, and the performance is impressive.
• I'm hesitant about adding a dependency on such a young package. Apecs currently depends only on well-known/widely used packages, the kind that stackage LTS waits for (rather than getting kicked out).
  • Maybe a better way for now is to release this as a standalone package (e.g. heph-apecs/apecs-heph), and link to it in the main apecs docs?
• Matching the Cache performance is wild. I don't remember if we ever had unboxed/storable caches (why not?), I wonder how those would've fared
• "memory usage scales with the highest entity ID" -- linearly?

[jtnuttall]

Thanks for the thoughtful feedback!

This was more of an RFC than a merge request, so I'm happy to close it.

I'm hesitant about adding a dependency on such a young package...

This is a totally fair point. Your suggestion of a standalone package seems like a great compromise. Would the standalone package be something you'd envision living in the apecs GitHub org, or would I maintain it under my own account for now?

"memory usage scales with the highest entity ID" -- linearly?

That's right. This is the main draw for this PR and the primary reason I would not recommend merging it as-is. Specifically, the overhead is on the order of 1.5 * max_entity_id words -- this will grow monotonically as the EntityCounter grows.

It is fixable, but it's a non-trivial change. The typical mitigation strategy (roughly how Bevy handles it) involves altering the EntityCounter substantially:

1. Tag each entity ID with a generation to invalidate stale lookups.
2. Maintain a pool of discarded entity IDs to be reused, keeping max_entity_id from growing infinitely.
3. When a "large enough" ID is reclaimed, trigger compact to shrink the backing arrays.

Matching the Cache performance is wild... I wonder how those would've fared

I'd speculate that a dedicated unboxed/storable Cache would perform within noise of the equivalent sparse set variants. I have a strong hunch, however, that the sparse-set approach would have a significant advantage on deletes, which I didn't benchmark.

Just as a final bit of context, my original motivation for this work also included benefits I didn't detail in the PR, like the great cache locality of sparse sets on intersections and faster parent/child queries, which can make 3D scene graphs feasible within the ECS itself.

[jonascarpay]

Would the standalone package be something you'd envision living in the apecs GitHub org, or would I maintain it under my own account for now?

Probably just your own account, but I'm thinking we add a "third-party packages" section to the readme where we link to it

The typical mitigation strategy (roughly how Bevy handles it) involves altering the EntityCounter substantially:

Interesting, I wonder if we could do something like that. Having generational entities seems nice.

[dpwiz]

I like entity ids being transparent so I can attach components to random external stuff like AST nodes or ProcessIDs.

[jonascarpay]

Yeah, we're definitely not changing the entity logic in the core apecs library

[jtnuttall]

Alright, thanks for taking the time to review this! I'll let you know if/when I get the time to create a spinoff package. I'll close this out now.