Conversation
Fixed ArgumentOutOfRangeException when performing matrix multiplication
on arrays with more than 2 dimensions (e.g., (3,1,2,2) @ (3,2,2)).
Root causes:
1. Default.MatMul.cs: Loop count used `l.size` (total elements) instead
of `iterShape.size` (number of matrix pairs to multiply)
2. UnmanagedStorage.Getters.cs: When indexing into broadcast arrays:
- sliceSize incorrectly used parent's BufferSize for non-broadcast
subshapes instead of the subshape's actual size
- Shape offset was double-counted (once in GetSubshape, again because
InternalArray.Slice already positioned at offset)
The fix ensures:
- Correct iteration count over batch dimensions
- Proper sliceSize calculation based on subshape broadcast status
- Shape offset reset to 0 after array slicing
Verified against NumPy 2.4.2 output.
The tests incorrectly expected both arrays to have IsBroadcasted=True after np.broadcast_arrays(). Per NumPy semantics, only arrays that actually get broadcasted (have stride=0 for dimensions with size>1) should be flagged. When broadcasting (1,1,1) with (1,10,1): - Array 'a' (1,1,1→1,10,1): IsBroadcasted=True (strides become 0) - Array 'b' (1,10,1→1,10,1): IsBroadcasted=False (no change, no zero strides) NumSharp's behavior was correct; the test expectations were wrong.
When np.sum() or np.mean() is called with keepdims=True and no axis specified (element-wise reduction), the result should preserve all dimensions as size 1. Before: np.sum(arr_2d, keepdims=True).shape = (1) After: np.sum(arr_2d, keepdims=True).shape = (1, 1) Fixed in both ReduceAdd and ReduceMean by reshaping to an array of 1s with the same number of dimensions as the input, instead of just calling ExpandDimension(0) once. Verified against NumPy 2.4.2 behavior.
Extended the keepdims fix to all remaining reduction operations: - ReduceAMax (np.amax, np.max) - ReduceAMin (np.amin, np.min) - ReduceProduct (np.prod) - ReduceStd (np.std) - ReduceVar (np.var) Also fixed np.amax/np.amin API layer which ignored keepdims when axis=null. Added comprehensive parameterized test covering all reductions with multiple dtypes (Int32, Int64, Single, Double, Int16, Byte) to prevent regression. All 7 reduction functions now correctly preserve dimensions with keepdims=true, matching NumPy 2.x behavior.
This commit introduces a dynamic IL code generation system for NumSharp's element-wise operations, replacing hundreds of thousands of lines of template-generated type-switch code with ~7K lines of IL emission logic. Architecture: - ILKernelGenerator.cs: Main IL emission engine (~4.5K lines) - Generates typed kernels at runtime via System.Reflection.Emit - SIMD vectorization for contiguous float/double arrays (Vector256) - Strided path for sliced/broadcast arrays via coordinate iteration - BinaryKernel.cs: Binary operation definitions (Add, Sub, Mul, Div, Mod) - UnaryKernel.cs: Unary operations (22 ops: Sin, Cos, Sqrt, Exp, etc.) - ReductionKernel.cs: Element-wise reductions (Sum, Prod, Max, Min, etc.) - ScalarKernel.cs: Scalar operation keys (eliminates dynamic dispatch) Dispatch files (DefaultEngine.*.cs): - BinaryOp.cs: Binary operation dispatch with type promotion - UnaryOp.cs: Unary operation dispatch - BitwiseOp.cs: Bitwise AND/OR/XOR (fixes broken & and | operators) - CompareOp.cs: Comparison operations (==, !=, <, >, <=, >=) - ReductionOp.cs: Element-wise reduction dispatch Bug fixes included: 1. operator & and operator | were completely broken (returned null) 2. Default.Log1p was incorrectly using Log10 instead of Log 3. Sliced array × scalar incorrectly used SIMD path (wrong indexing) 4. Division type promotion: int/int now returns float64 (NumPy 2.x) 5. Sign(NaN) threw ArithmeticException, now returns NaN Files deleted: 73 type-specific generated files (~500K lines) - Add/*.cs, Subtract/*.cs, Multiply/*.cs, Divide/*.cs, Mod/*.cs (60 files) - Equals/*.cs (13 files) Files simplified: 22 unary operation files now single-line delegations Test results: 2597 tests pass (excluding 11 skipped, OpenBugs excluded)
Nucs
added
bug
Implement Vector256 SIMD operations for mixed-type scalar operations where the array type equals the result type (no per-element conversion needed). This optimizes operations like `double_array + int_scalar`. ## Changes - Add `EmitSimdScalarRightLoop()` for SIMD scalar right operand - Add `EmitSimdScalarLeftLoop()` for SIMD scalar left operand - Add `EmitVectorCreate()` helper for Vector256.Create(scalar) - Update `GenerateSimdScalarRightKernel()` to choose SIMD when eligible - Update `GenerateSimdScalarLeftKernel()` to choose SIMD when eligible ## SIMD Eligibility SIMD is used when: - ScalarRight: `LhsType == ResultType` (array needs no conversion) - ScalarLeft: `RhsType == ResultType` (array needs no conversion) - ResultType supports SIMD (float, double, int, long, etc.) - Operation has SIMD support (Add, Subtract, Multiply, Divide) ## Benchmark Results Array size: 10,000,000 elements Before (mixed-type used scalar loop): int + double_scalar: 19.09 ms After (SIMD when eligible): double + int_scalar: 14.96 ms [IL SIMD - matches baseline] float + int_scalar: 7.18 ms [IL SIMD - matches baseline] int + double_scalar: 15.84 ms [still scalar - needs conversion] ## Technical Details The SIMD scalar loop: 1. Loads scalar, converts to result type if needed 2. Broadcasts scalar to Vector256 using Vector256.Create() 3. SIMD loop: load array vector, perform vector op, store result 4. Tail loop handles remainder elements All 2597 tests pass.
…Py comparison Benchmark Runner (run-benchmarks.ps1): - Added -Experimental flag for research benchmarks (dispatch, fusion) - Added timestamped results archiving to results/yyyyMMdd-HHmmss/ - Added comprehensive logging with Write-Log function to benchmark.log - Fixed -Type and -Quick filters for C# BDN benchmarks - Filter order changed from *DType:*N:* to *N:*DType:* to match BDN parameter output - JSON naming: numpy-results.json for Python, benchmark-report.json for merged - Added suite validation with helpful error messages Python Benchmarks (numpy_benchmark.py): - Added all 16 arithmetic operations per dtype to match C# benchmarks: - Add: element-wise, np.add, scalar, literal (4 ops) - Subtract: element-wise, scalar, scalar-left (3 ops) - Multiply: element-wise, square, scalar, literal (4 ops) - Divide: element-wise, scalar, scalar-left (3 ops) - Modulo: element-wise, literal (2 ops) - Operations run for CommonTypes: int32, int64, float32, float64 Merge Script (merge-results.py): - Fixed normalize_op_name regex to only strip dtype suffixes like (int32) - Previously stripped any parentheses including (element-wise) descriptors - Added comprehensive method name mappings for all operations C# Benchmarks: - Fixed CreatePositiveArray to return non-zero values for ALL types - Previously only handled floats, caused DivideByZeroException for int modulo Results (64 operations benchmarked): - 61 operations faster than NumPy (95%) - 3 operations within 2x of NumPy - Modulo operations 2-10x faster than NumPy - Most arithmetic operations 0.6-0.9x (10-40% faster)
Benchmarks comparing allocation strategies: - AllocationMicroBenchmarks: Marshal.AllocHGlobal vs NativeMemory.Alloc/AlignedAlloc - ZeroInitBenchmarks: Alloc+InitBlock vs AllocZeroed for np.zeros optimization - NumSharpAllocationBenchmarks: End-to-end NumSharp array creation impact - AllocationSizeBenchmarks: Size scaling from 64B to 64MB Menu option 13 added for allocation benchmarks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make IL kernel generator adaptive to available hardware SIMD support by detecting Vector512/256/128 at runtime instead of hardcoding Vector256. - Add VectorBits/VectorBytes static properties for hardware detection - Refactor GetVectorCount, EmitVectorLoad/Store/Operation to use detected width - Remove obsolete UmanagedArrayTests.cs (typo in filename, tests covered elsewhere) - Normalize line endings in csproj files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Normalize all text files to LF in the repository to prevent CRLF/LF inconsistencies between Windows and WSL development. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Apply .gitattributes normalization across all text files. No code changes - only CRLF → LF conversion. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace deprecated Marshal.AllocHGlobal/FreeHGlobal with the modern .NET 6+ NativeMemory.Alloc/Free API. Benchmarks confirmed identical performance (within noise margin). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the 5,183-line ILKernelGenerator.cs into 6 focused partial class files: - ILKernelGenerator.cs (507 lines): Core constants, vector helpers, NPTypeCode IL helpers - ILKernelGenerator.Binary.cs (443 lines): Binary same-type operations - ILKernelGenerator.MixedType.cs (1,064 lines): Mixed-type binary operations - ILKernelGenerator.Unary.cs (1,199 lines): Unary operations + scalar kernels - ILKernelGenerator.Comparison.cs (964 lines): Comparison operations - ILKernelGenerator.Reduction.cs (1,073 lines): Reduction operations This improves maintainability by organizing code by kernel type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
7 tasks
Member
Author
The enum value name was misleading after migrating from Marshal.AllocHGlobal to NativeMemory.Alloc. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The comment incorrectly stated ALIGNED is true because of "managed memory". The actual reason is that NumSharp uses Vector.Load (unaligned) rather than Vector.LoadAligned, so alignment tracking is unnecessary. Closes #581 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements the IL Kernel Generator, replacing NumSharp's ~500K+ lines of template-generated type-switch code with ~7K lines of dynamic IL emission using
System.Reflection.Emit.Closes #544 - [Core] Replace ~636K lines of generated math code with DynamicMethod IL emission
Closes #545 - [Core] SIMD-Optimized IL Emission (SIMD for contiguous arrays AND scalar broadcast)
Changes
Core Kernel Infrastructure (~7K lines)
ILKernelGenerator.csSimdKernels.csReductionKernel.csBinaryKernel.csDispatch Files
DefaultEngine.BinaryOp.cs- Binary ops (Add, Sub, Mul, Div, Mod)DefaultEngine.UnaryOp.cs- 22 unary ops (Sin, Cos, Sqrt, Exp, etc.)DefaultEngine.CompareOp.cs- Comparisons (==, !=, <, >, <=, >=)DefaultEngine.BitwiseOp.cs- Bitwise AND/OR/XORDefaultEngine.ReductionOp.cs- Element-wise reductionsFiles Deleted (73 total)
Net change: -498,481 lines (13,553 additions, 512,034 deletions)
SIMD Optimizations
Execution Path SIMD Status
Note: Same-type operations (e.g.,
double + double) fall back to C#SimdKernels.cswhich has full SIMD for SimdFull, SimdScalarRight/Left, and SimdChunk paths.Scalar Broadcast Optimization
SIMD scalar operations hoist
Vector256.Create(scalar)outside the loop:Benchmark (10M elements):
Bug Fixes Included
operator &andoperator |- Were completely broken (returned null)Log1p- Incorrectly usingLog10instead ofLogint/intnow returnsfloat64(NumPy 2.x behavior)Sign(NaN)- Now returns NaN instead of throwingArithmeticExceptionTest Plan
Architecture
Performance
Future Work
int + double_scalarcasesAdditional: NativeMemory Modernization
Closes #528 - Modernize unmanaged allocation: Marshal.AllocHGlobal → NativeMemory
Replaced deprecated
Marshal.AllocHGlobal/FreeHGlobalwith modern .NET 6+NativeMemory.Alloc/FreeAPI across 5 allocation sites. Benchmarks confirmed identical performance.