Explorer: performance overhaul, stream preview, and e2e testing

[benbernard]

Summary

• Replace structuredClone with fast JSON clone and sample-based estimateSize (~4x pipeline speedup)
• Stabilize wrappedDispatch with useRef and remove unused useState from useExecution to eliminate full-screen re-renders
• Memoize 8 components with React.memo
• Add stream preview to AddStageModal and EditStageModal (Tab to toggle panels, Enter to zoom into records)
• Split useInput hooks to fix typing garble when TextInput is active
• Fix fuzzy search ranking with name-first matching (+200 bonus) and minScore threshold
• Fix fromps race condition, undo/redo stale cache, execution retry after error, disabled first stage, welcome screen
• Rename tui → explorer throughout codebase
• Add tmux-based e2e test harness with 10 smoke tests (RUN_E2E=1 bun test tests/explorer/e2e/)
• Add 560+ unit tests: executor, reducer, fuzzy-match, unicode, cache-manager, sessions, etc.
• Add executor benchmark script

Test plan

• All 1383 tests pass (12 skipped: tmux e2e and slow fromps tests)
• Lint and typecheck clean
• Manual tmux smoke tests: npm run test:e2e

🤖 Generated with Claude Code

[github-actions]

Performance Benchmark Results

⚠️ 10 regressions detected out of 103 benchmarks (threshold: 25%)

Benchmark	Median	Baseline	Delta
Compiled KeySpec.setValue — nested (address/zip)	220.8µs	156.6µs	+41.0% 🔴
grep — 10K records (string match)	828.1µs	443.6µs	+86.7% 🔴
sort — 100 records (by score, numeric)	216.3µs	147.4µs	+46.8% 🔴
chain — 5 ops (grep	eval	grep	eval
new Record() — 10K objects	507.1µs	149.3µs	+239.6% 🔴
Record.get — 10K records × 3 fields	63.1µs	47.7µs	+32.2% 🔴
implicit — 3 ops (grep	eval	grep), 1K records	346.0µs
implicit — 5 ops (grep	eval	grep	eval
TextDecoderStream — 10K lines	5.23ms	4.04ms	+29.6% 🔴
bulk text + split — 100K lines	33.55ms	25.55ms	+31.3% 🔴

103 benchmarks: 17 faster, 19 slower, 67 within noise (10%)

ℹ️ Note: Benchmarks are advisory-only. GitHub Actions shared runners have variable performance, so results may fluctuate ±25% between runs. For reliable benchmarking, run locally with bun run bench.

Full benchmark results

JSON Parsing

Benchmark	Median	Baseline	Delta	Throughput
Record.fromJSON — 100 lines	159.4µs	153.2µs	+4.1%	627.40K rec/s
Record.fromJSON — 10K lines	14.52ms	13.09ms	+10.9% 🔴	688.54K rec/s, 203.9 MB/s
InputStream.fromString — 100 records	220.2µs	214.6µs	+2.6%	454.13K rec/s
InputStream.fromString — 10K records	19.26ms	18.15ms	+6.2%	519.15K rec/s, 153.7 MB/s
JSON.parse baseline — 10K lines (no Record)	13.22ms	12.79ms	+3.4%	756.26K rec/s, 224.0 MB/s
JSON.parse single array — 10K records	12.98ms	12.50ms	+3.8%	770.48K rec/s, 228.2 MB/s

JSON Serialization

Benchmark	Median	Baseline	Delta	Throughput
Record.toString — 100 records	82.1µs	86.6µs	-5.2%	1.22M rec/s
Record.toString — 10K records	8.56ms	8.10ms	+5.7%	1.17M rec/s, 346.1 MB/s
Record.toJSON — 10K records	293.4µs	279.0µs	+5.2%	34.08M rec/s
JSON.stringify baseline — 10K objects (no Record)	8.05ms	7.92ms	+1.6%	1.24M rec/s, 368.0 MB/s
Batch join — 10K records (map+join)	8.82ms	8.27ms	+6.6%	1.13M rec/s, 335.8 MB/s

KeySpec Access

Benchmark	Median	Baseline	Delta	Throughput
KeySpec — simple key (name)	218.2µs	214.4µs	+1.8%	45.82M rec/s
KeySpec — nested key (address/zip)	542.9µs	530.2µs	+2.4%	18.42M rec/s
KeySpec — deep nested (address/coords/lat)	571.6µs	551.8µs	+3.6%	17.49M rec/s
KeySpec — array index (tags/#0)	545.9µs	528.8µs	+3.2%	18.32M rec/s
Direct property access baseline (rec['name'])	44.1µs	52.4µs	-15.8% 🟢	226.59M rec/s
Direct nested access baseline (rec.address.coords.lat)	82.9µs	100.9µs	-17.8% 🟢	120.63M rec/s
KeySpec construction — cached (same spec 10K times)	281.9µs	295.2µs	-4.5%	35.48M rec/s
KeySpec construction — unique specs (10K different)	2.75ms	3.38ms	-18.6% 🟢	3.63M rec/s
Compiled KeySpec.resolveValue — nested (address/zip)	165.3µs	312.4µs	-47.1% 🟢	60.50M rec/s
Compiled KeySpec.resolveValue — deep (address/coords/lat)	123.9µs	176.9µs	-30.0% 🟢	80.72M rec/s
Compiled KeySpec.resolveValue — array (tags/#0)	154.1µs	267.2µs	-42.3% 🟢	64.88M rec/s
Compiled KeySpec.setValue — nested (address/zip)	220.8µs	156.6µs	+41.0% 🔴	45.28M rec/s

Core Operations

Benchmark	Median	Baseline	Delta	Throughput
grep — 10K records (r.age > 50)	449.4µs	447.3µs	+0.5%	22.25M rec/s
grep — 10K records (string match)	828.1µs	443.6µs	+86.7% 🔴	12.08M rec/s
eval — 10K records (add computed field)	2.51ms	2.41ms	+4.3%	3.99M rec/s
xform — 10K records (push each record)	2.45ms	2.81ms	-12.7% 🟢	4.08M rec/s
sort — 100 records (by score, numeric)	216.3µs	147.4µs	+46.8% 🔴	462.22K rec/s
sort — 10K records (by score, numeric)	19.29ms	18.48ms	+4.4%	518.38K rec/s
sort — 10K records (by name, lexical)	12.71ms	12.27ms	+3.6%	786.87K rec/s
collate — 100 records (count by city)	391.9µs	362.5µs	+8.1%	255.20K rec/s
collate — 10K records (count by city)	13.81ms	11.77ms	+17.3% 🔴	724.20K rec/s
fromcsv — 10K rows (parse CSV to records)	14.50ms	15.04ms	-3.6%	689.86K rec/s, 45.3 MB/s

Pipeline Overhead

Benchmark	Median	Baseline	Delta	Throughput
chain — single op (grep), 10K records	7.49ms	7.04ms	+6.3%	1.34M rec/s
chain — 3 ops (grep	eval	grep), 10K records	8.17ms	7.27ms
chain — 5 ops (grep	eval	grep	eval	grep), 10K records
passthrough baseline — 10K records (direct collector)	6.31ms	5.97ms	+5.6%	1.59M rec/s

Record Creation & Serialization

Benchmark	Median	Baseline	Delta	Throughput
new Record() — 10K objects	507.1µs	149.3µs	+239.6% 🔴	19.72M rec/s
new Record() empty — 10K	151.4µs	135.5µs	+11.7% 🔴	66.07M rec/s
Record.get — 10K records × 3 fields	63.1µs	47.7µs	+32.2% 🔴	475.79M rec/s
Record.set — 10K records × 1 field	67.8µs	60.8µs	+11.6% 🔴	147.45M rec/s
Record.toJSON — 10K records	285.1µs	279.0µs	+2.2%	35.08M rec/s
Record.toString — 10K records	8.76ms	8.10ms	+8.2%	1.14M rec/s
Record.clone — 10K records	6.37ms	56.23ms	-88.7% 🟢	1.57M rec/s
Record.fromJSON — 10K lines	13.90ms	13.09ms	+6.2%	719.48K rec/s, 213.1 MB/s
Record.dataRef — 10K records (zero-copy)	40.1µs	38.2µs	+4.9%	249.47M rec/s
Record.sort — 10K records (numeric field)	11.83ms	11.49ms	+3.0%	845.29K rec/s
Record.sort — 10K records (lexical field)	6.19ms	6.09ms	+1.7%	1.62M rec/s
Record.cmp — 1M comparisons (single field)	104.83ms	112.05ms	-6.4%	9.54M rec/s
Record.sort — 10K records (nested field numeric)	16.06ms	15.62ms	+2.9%	622.48K rec/s
Record.cmp — 1M comparisons (multi-field cached)	91.41ms	87.21ms	+4.8%	10.94M rec/s
Record.sort — 10K records (cached comparator reuse)	11.66ms	11.74ms	-0.7%	857.76K rec/s

Chain vs Pipe

Benchmark	Median	Baseline	Delta	Throughput
chain — 2 ops (grep	eval), 100 records	132.1µs	153.0µs	-13.7% 🟢
pipe — 2 ops (grep	eval), 100 records	450.51ms	419.85ms	+7.3%
implicit — 2 ops (grep	eval), 100 records	115.6µs	144.1µs	-19.8% 🟢
chain — 2 ops (grep	eval), 1K records	330.0µs	301.8µs	+9.3%
pipe — 2 ops (grep	eval), 1K records	393.17ms	384.03ms	+2.4%
implicit — 2 ops (grep	eval), 1K records	216.0µs	209.2µs	+3.3%
chain — 2 ops (grep	eval), 10K records	1.11ms	1.07ms	+3.4%
pipe — 2 ops (grep	eval), 10K records	376.96ms	396.94ms	-5.0%
implicit — 2 ops (grep	eval), 10K records	1.14ms	1.22ms	-6.4%
chain — 3 ops (grep	eval	grep), 100 records	149.3µs	159.5µs
pipe — 3 ops (grep	eval	grep), 100 records	566.74ms	546.67ms
implicit — 3 ops (grep	eval	grep), 100 records	134.3µs	162.2µs
chain — 3 ops (grep	eval	grep), 1K records	220.2µs	203.0µs
pipe — 3 ops (grep	eval	grep), 1K records	563.20ms	578.88ms
implicit — 3 ops (grep	eval	grep), 1K records	346.0µs	218.9µs
chain — 3 ops (grep	eval	grep), 10K records	1.13ms	1.06ms
pipe — 3 ops (grep	eval	grep), 10K records	570.96ms	564.71ms
implicit — 3 ops (grep	eval	grep), 10K records	1.17ms	1.40ms
chain — 5 ops (grep	eval	grep	eval	grep), 100 records
pipe — 5 ops (grep	eval	grep	eval	grep), 100 records
implicit — 5 ops (grep	eval	grep	eval	grep), 100 records
chain — 5 ops (grep	eval	grep	eval	grep), 1K records
pipe — 5 ops (grep	eval	grep	eval	grep), 1K records
implicit — 5 ops (grep	eval	grep	eval	grep), 1K records
chain — 5 ops (grep	eval	grep	eval	grep), 10K records
pipe — 5 ops (grep	eval	grep	eval	grep), 10K records
implicit — 5 ops (grep	eval	grep	eval	grep), 10K records

Line Reading

Benchmark	Median	Baseline	Delta	Throughput
InputStream.fromFile — 100 lines	518.5µs	513.2µs	+1.0%	192.85K rec/s, 57.0 MB/s
InputStream.fromString — 100 lines	195.6µs	192.9µs	+1.4%	511.32K rec/s, 151.0 MB/s
manual buffer (isolated) — 100 lines	264.5µs	266.4µs	-0.7%	378.02K rec/s, 111.7 MB/s
bulk text + split — 100 lines	96.6µs	78.0µs	+23.8% 🔴	1.04M rec/s, 305.9 MB/s
node readline — 100 lines	479.9µs	513.3µs	-6.5%	208.37K rec/s, 61.5 MB/s
TextDecoderStream — 100 lines	279.6µs	277.7µs	+0.7%	357.68K rec/s, 105.6 MB/s
binary newline scan — 100 lines	287.4µs	368.1µs	-21.9% 🟢	347.99K rec/s, 102.8 MB/s
bun native stdin — 100 lines	26.81ms	24.93ms	+7.5%	3.73K rec/s, 1.1 MB/s
InputStream.fromFile — 10K lines	24.94ms	23.85ms	+4.6%	400.92K rec/s, 118.7 MB/s
InputStream.fromString — 10K lines	18.99ms	17.71ms	+7.2%	526.70K rec/s, 156.0 MB/s
manual buffer (isolated) — 10K lines	6.58ms	6.27ms	+5.0%	1.52M rec/s, 450.0 MB/s
bulk text + split — 10K lines	2.48ms	2.37ms	+4.8%	4.03M rec/s, 1194.1 MB/s
node readline — 10K lines	9.94ms	9.33ms	+6.6%	1.01M rec/s, 297.8 MB/s
TextDecoderStream — 10K lines	5.23ms	4.04ms	+29.6% 🔴	1.91M rec/s, 565.9 MB/s
binary newline scan — 10K lines	7.92ms	9.85ms	-19.6% 🟢	1.26M rec/s, 374.0 MB/s
bun native stdin — 10K lines	44.48ms	41.98ms	+5.9%	224.84K rec/s, 66.6 MB/s
InputStream.fromFile — 100K lines	283.89ms	258.30ms	+9.9%	352.25K rec/s, 104.7 MB/s
InputStream.fromString — 100K lines	229.04ms	220.03ms	+4.1%	436.60K rec/s, 129.7 MB/s
manual buffer (isolated) — 100K lines	31.13ms	36.57ms	-14.9% 🟢	3.21M rec/s, 954.7 MB/s
bulk text + split — 100K lines	33.55ms	25.55ms	+31.3% 🔴	2.98M rec/s, 885.7 MB/s
node readline — 100K lines	90.18ms	85.49ms	+5.5%	1.11M rec/s, 329.5 MB/s
TextDecoderStream — 100K lines	37.48ms	36.52ms	+2.6%	2.67M rec/s, 792.9 MB/s
binary newline scan — 100K lines	73.16ms	77.17ms	-5.2%	1.37M rec/s, 406.2 MB/s
bun native stdin — 100K lines	131.44ms	113.90ms	+15.4% 🔴	760.83K rec/s, 226.1 MB/s