feat(muon-optimizer): Add Muon optimizer#3
Merged
cristianglezm merged 5 commits intomasterfrom Jan 27, 2026
Merged
Conversation
… tests Introduce the Muon optimizer, a hybrid momentum + RMS-based algorithm. This patch adds: - New Muon optimizer (header + source) with JSON serialization/deserialization - Integration into Optimizers.hpp - Weight‑update logic using RMS scaling and momentum - zeroGrad implementation - Unit test validating behavior, update direction, and JSON round‑trip - Minor formatting cleanup in tests - Increase version from 1.0.0 to 1.1.0 Provides a third built‑in optimizer alongside SGD and Adam.
cristianglezm
force-pushed
the
feat/muon-optimizer
branch
6 times, most recently
from
e5f82e2 to
68c7051
Compare
- Add tag-based release workflow (trigger on v* tags) - Add Release-only packaging and artifact publishing - Add full platform matrix: * Windows (MinGW GCC, MinGW Clang) * Linux (GCC, Clang) * macOS (x64, arm64, Xcode) * Android (x86, x86_64, armeabi-v7a, arm64-v8a) * Emscripten (Static) - Unify CMake build flow across all platforms - Run tests on all Debug/Release builds - Run install + CPack only for Release builds - Add caching for: * JsonBox + googletest dependencies * Android NDK * Emscripten SDK - Add MSYS2 setup for Windows builds to fix pathing issues - Add Android SDK/NDK installation on cache miss - Add Emscripten activation and Node test runner - Remove old platform-specific build logic - Remove duplicated flags from matrix entries - Add release job to create GitHub releases with artifacts
cristianglezm
force-pushed
the
feat/muon-optimizer
branch
from
68c7051 to
91a8da9
Compare
…ong test expectations
This patch fixes several sources of undefined behavior that were exposed on
Windows STL debug builds:
- createFeedForwardNN:
- Added validation for numNeuronsPerHidden.size() < numHidden.
- Corrected comments and clarified first-hidden-layer connection.
- Fixed last hidden → output connection to use proper layer indices and sizes.
- createElmanNeuralNetwork:
- Rewrote layer indexing logic to correctly map input/context/hidden/output.
- Removed all invalid uses of numNeuronsPerHiddenLayer[i] where `i` was a
network-layer index instead of a hidden-layer index.
- Fixed forward and recurrent connections to avoid OOB access.
- Cleaned up bias initialization loops.
- MultiStepLR:
- Fixed UB by checking m_counter < m_steps.size() before indexing m_steps.
- Ensures scheduler no longer reads past the end of the steps vector.
- NeuralNetworkTest:
- Removed invalid w2 access (nnRef[0][0][1]) which never existed.
- Corrected gradient comment and expectation to use nnRef[0][1][0] as w2.
These fixes eliminate several silent OOB reads that only surfaced under
Windows MinGW / clang debug iterators, improving portability and correctness.
This updates all CI matrix entries to use newer GitHub-hosted runners: - Windows: 2022 → 2025 - Linux: ubuntu-22.04 → ubuntu-24.04 - macOS: macos-12/14 → macos-15 - Android builds now run on ubuntu-24.04 instead of ubuntu-22.04 - Emscripten also moved to ubuntu-24.04 No logic changes to the build steps; only runner OS versions were bumped to keep CI aligned with current GitHub Actions environments.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce the Muon optimizer, a hybrid momentum + RMS-based algorithm.
This patch adds:
- New Muon optimizer (header + source) with JSON serialization/deserialization
- Integration into Optimizers.hpp
- Weight‑update logic using RMS scaling and momentum
- zeroGrad implementation
- Unit test validating behavior, update direction, and JSON round‑trip
- Minor formatting cleanup in tests
- Increase version from 1.0.0 to 1.1.0