Skip to content

Conversation

@naftaly
Copy link
Owner

@naftaly naftaly commented Nov 17, 2025

Summary

This PR introduces BasicType - a type-safe wrapper system for memory-mapped storage - along with major performance optimizations and architectural improvements to the MemoryMap library.

Key Features

  • BasicType System: Type-safe containers for Int, String, Double, Bool with three size variants:

    • BasicType8 (BasicTypeNumber): 10 bytes - optimized for numeric types
    • BasicType64: 62 bytes - general purpose with short strings (~60 chars)
    • BasicType1024: 1022 bytes - for longer strings (~1000 chars)
  • Lock Abstraction: Flexible locking via MemoryMapLock protocol

    • OSAllocatedUnfairLock (default) - high performance
    • NoLock - zero-cost for single-threaded use
    • NSLock - standard Foundation lock
    • Custom lock support
  • Performance Optimizations:

    • ~11-12μs per KeyValueStore lookup, consistent across 25%-99% load factors
    • Direct buffer access via withUnsafeBytes eliminates subscript overhead
    • @inline(__always) on hot paths
    • Double hashing with FNV-1a for excellent distribution
  • Architectural Improvements:

    • ByteStorage layer with tuple-based storage maintains POD compliance
    • KeyValueStore+Internal.swift separates implementation details
    • Minimized disk writes with direct field modification patterns

Breaking Changes

KeyValueStore API Change: Now requires Key and Value type parameters

// Before
let store = try KeyValueStore<UserData>(fileURL: url)

// After  
let store = try KeyValueStore<BasicType64, UserData>(fileURL: url)
store["user:123"] = userData  // String literals work automatically

Testing & CI

  • New test suites: BasicType benchmarks, cache locality, capacity comparisons, mixed operations
  • Enhanced CI: New test.yml workflow for multi-platform testing (macOS, iOS, tvOS, watchOS, visionOS)
  • Improved benchmarks: Detailed performance reporting across core operations, load factors, bulk operations, stress tests

Test Plan

  • All existing tests pass
  • New BasicType tests verify correctness across all supported types
  • Load factor benchmarks confirm consistent performance (11-12μs) from 25%-99% capacity
  • Multi-platform CI validates macOS, iOS, tvOS, watchOS, visionOS support
  • Performance benchmarks show <2.5ms for 200 lookups (main-thread safe)
  • Lock abstraction tested with NoLock, NSLock, default lock

Repository owner deleted a comment from github-actions bot Nov 18, 2025
Repository owner deleted a comment from github-actions bot Nov 18, 2025
Repository owner deleted a comment from github-actions bot Nov 19, 2025
Repository owner deleted a comment from github-actions bot Nov 19, 2025
Repository owner deleted a comment from github-actions bot Nov 19, 2025
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

🚀 MemoryMap Performance Benchmarks

256-entry capacity with double hashing, @inline(__always) optimizations

Test Hardware: Apple M1 (Virtual), 7 GB RAM

Core Operations

Operation Time Per-Op Main Thread
Contains 1.0ms 10.0 μs ✅ Excellent
Insert 1.0ms 10.0 μs ✅ Excellent
Lookup (hit) 3.0ms 30.0 μs ✅ Excellent
Lookup (miss) 1.0ms 10.0 μs ✅ Excellent
Mixed operations 2.0ms 10.0 μs ✅ Excellent
Remove (insert+delete) 3.0ms 15.0 μs ✅ Excellent
Update 1.0ms 10.0 μs ✅ Excellent

Load Factor Performance

Load % Lookups Time Per-Lookup Status
25% 6,400 154ms 24.1 μs ✅ Good
50% 12,800 265ms 20.7 μs ✅ Good
75% 19,200 476ms 24.8 μs ✅ Good
90% 23,000 511ms 22.2 μs ✅ Good
99% 25,300 591ms 23.4 μs ✅ Good

Key Length Impact (100 ops)

Key Length Time Per-Op
Long (64 chars) 3.0ms 30.0 μs
Medium (~25 chars) 6.0ms 60.0 μs
Short (2-3 chars) 2.0ms 20.0 μs

Bulk Operations

Operation Time Description
Count (100 entries) 4.0ms ✅ Excellent
Keys iteration (100 entries) 0.0ms ✅ Excellent
Large batch write 3.0ms ✅ Excellent
Remove all entries 1.0ms ✅ Excellent
Convert to Dictionary (100 entries) 0.0ms ✅ Excellent

Stress & Edge Cases

Test Time Status
Many tombstones 1ms ✅ Good
Random access pattern 2ms ✅ Good
Sequential vs random access 2ms ✅ Good
Worst-case probe chain 103ms ❌ Slow

Persistence

Operation Time Status
Write, close, reopen 1ms ✅ Excellent

Performance Characteristics

Main Thread Budget

  • Excellent: <10ms - Perfect for UI interactions
  • Good: 10-50ms - Acceptable for most operations
  • ⚠️ OK: 50-100ms - Use with caution on main thread
  • Review: >100ms - Consider background thread

Target: 16.67ms/frame @ 60fps, 8.33ms/frame @ 120fps

Capacity & Optimization

  • Fixed capacity: 256 entries
  • Recommended usage: ≤200 keys for optimal performance
  • Memory footprint: ~306KB per store
  • Key optimizations: Double hashing, @inline(__always), direct buffer access via withUnsafeBytes

Total tests: 25 passed | Generated 2025-11-19 20:50:26

@naftaly naftaly changed the title Basic Type Add BasicType system with type-safe storage and configurable locking Nov 19, 2025
@naftaly naftaly marked this pull request as ready for review November 19, 2025 20:46
@naftaly naftaly merged commit f1ee648 into main Nov 19, 2025
6 checks passed
@naftaly naftaly deleted the acohen/basic-type branch November 19, 2025 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants