🙏 Add OmniRoute to README — A Fork That Grew Thanks to 9Router#151
🙏 Add OmniRoute to README — A Fork That Grew Thanks to 9Router#151diegosouzapw wants to merge 5 commits intodecolua:masterfrom
Conversation
Bug fixes: - Fix semaphore slot leak in embeddings/images routes (4 files) - Fix isModelAvailable pre-check leaking semaphore slots - Fix broken function identity comparison in auth.js - Add markRateLimited() for account-level 429s Documentation: - Add dual-layer rate limiting architecture section to ARCHITECTURE.md - Add rateLimitManager.js to module listing - Update README with combo and semaphore docs - Update profile page UI (concurrencyPerAccount + queueTimeoutMs) - Replace stickyRoundRobinLimit with semaphore-based concurrency
fix(auth): semaphore leak fixes + docs(arch): dual-layer rate limiting
- Merge 'Local Mode' and 'Data & Backups' into new 'System & Storage' card - Reorder cards: System & Storage → Security → Routing → Combo → Proxy → Appearance - Add 'Backup Now' button for manual SQLite backup trigger - Show last backup timestamp and database size in Storage card - Add PUT /api/db-backups endpoint for manual backup creation - Update backup description text to reflect SQLite rotation policy - Remove standalone 'Local Mode' card (info merged into System & Storage)
feat(profile): redesign profile page layout and add manual backup
There was a problem hiding this comment.
Pull request overview
This PR introduces substantial new routing/concurrency, dashboard, and documentation changes (despite the PR metadata focusing on adding OmniRoute to the README).
Changes:
- Reworks account selection to use a round-robin counter plus (intended) semaphore-based per-account concurrency gating.
- Expands chat combo handling to pass richer combo/config context into the shared combo handler.
- Adds dashboard UI and API scaffolding for backups, global proxy status, and “combo defaults”, plus updates README/architecture docs to describe these capabilities.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| src/sse/services/auth.js | Adds RR counters and attempts to add semaphore-based per-account concurrency + cooldown integration. |
| src/sse/handlers/chat.js | Refactors combo detection/dispatch and adds a model availability pre-check + extra combo config plumbing. |
| src/lib/localDb.js | Updates default settings schema (adds concurrency settings; removes sticky RR limit default). |
| src/app/api/db-backups/route.js | Introduces new API route for listing/creating/restoring DB backups (currently missing dependencies). |
| src/app/(dashboard)/dashboard/profile/page.js | Adds UI for backups/restore, combo defaults, and global proxy configuration/status (calls multiple missing APIs). |
| docs/ARCHITECTURE.md | Updates architecture documentation to describe combo strategies and rate-limiting layers (references missing modules). |
| README.md | Updates feature descriptions/tables around combos and semaphore-based rate limiting (does not add OmniRoute mention). |
| 9router-pr-description.md | Adds a standalone PR-description markdown file (non-runtime). |
Comments suppressed due to low confidence (9)
src/lib/localDb.js:53
stickyRoundRobinLimitwas removed from the default settings schema here, but parts of the dashboard still read/write it and the previous RR behavior depended on it. Either keep it in defaults for backward compatibility, or remove/rename it everywhere and ensure no UI/API continues to surface it.
settings: {
cloudEnabled: false,
concurrencyPerAccount: 3,
queueTimeoutMs: 30000,
docs/ARCHITECTURE.md:136
- This section lists several
open-sse/services/*modules (comboConfig.js,comboMetrics.js,rateLimitSemaphore.js,rateLimitManager.js) that are not present in theopen-sse/services/directory in this repo. Please either add the referenced modules in this PR or update the architecture doc to reflect the actual code layout.
- Combo strategies and orchestration: `open-sse/services/combo.js`
- Combo config cascade: `open-sse/services/comboConfig.js`
- Combo metrics tracking: `open-sse/services/comboMetrics.js`
- Rate-limit semaphore (per-model/account concurrency): `open-sse/services/rateLimitSemaphore.js`
- Adaptive rate-limit manager (Bottleneck, per-connection RPM): `open-sse/services/rateLimitManager.js`
src/sse/services/auth.js:102
- The new round-robin selection rotates on every call using an in-memory counter and no longer consults any sticky-per-account limit setting. If the dashboard/API still expose
stickyRoundRobinLimit, it becomes dead config. Either remove it across the UI/API, or reintroduce sticky behavior in this selection logic.
// Round-robin: circular selection using in-memory counter (no DB writes)
const counter = rrCounters.get(provider) || 0;
rrCounters.set(provider, counter + 1);
const index = counter % availableConnections.length;
connection = availableConnections[index];
README.md:37
- The PR title/description are about adding OmniRoute to the README, but the README changes here focus on new semaphore-based concurrency control, combo strategies, proxying, etc., and do not add any OmniRoute mention. Please align the PR scope: either add the requested OmniRoute README entry, or update the PR title/description to reflect the substantial functional changes being introduced.
- ✅ **Maximize subscriptions** - Track quota, use every bit before reset
- ✅ **Auto fallback** - Subscription → Cheap → Free, zero downtime
- ✅ **Multi-account** - Semaphore-based concurrency control per account per provider
- ✅ **Universal** - Works with Claude Code, Codex, Gemini CLI, Cursor, Cline, any CLI tool
src/app/(dashboard)/dashboard/profile/page.js:445
- These backup timestamps are formatted with a hard-coded locale (
"pt-BR"), which is inconsistent with the rest of the dashboard date formatting (generally uses default locale). Consider using the user's locale (toLocaleString()/toLocaleString(undefined, ...)) or a shared formatter for consistent output.
<div className="flex items-center gap-2 mb-1">
<span className="material-symbols-outlined text-[16px] text-amber-500">description</span>
<span className="text-sm font-medium truncate">
{new Date(backup.createdAt).toLocaleString("pt-BR")}
</span>
src/app/(dashboard)/dashboard/profile/page.js:723
- If
comboDefaults.healthCheckEnabledcan be missing/undefined (e.g., older stored configs), this toggle handler will always set it totrue(!undefined === true) and may not actually flip the effective value. Consider toggling based onprev.healthCheckEnabled !== falseso undefined behaves consistently.
<Toggle
checked={comboDefaults.healthCheckEnabled !== false}
onChange={() => setComboDefaults(prev => ({ ...prev, healthCheckEnabled: !prev.healthCheckEnabled }))}
/>
</div>
src/app/(dashboard)/dashboard/profile/page.js:732
- Same issue as above: if
comboDefaults.trackMetricsisundefined, this handler will set it totrueand may not flip the effective value (sincecheckedtreats undefined as enabled). Consider toggling based onprev.trackMetrics !== falsefor consistent behavior with missing fields.
</div>
<Toggle
checked={comboDefaults.trackMetrics !== false}
onChange={() => setComboDefaults(prev => ({ ...prev, trackMetrics: !prev.trackMetrics }))}
/>
src/sse/handlers/chat.js:94
- This availability pre-check calls
getProviderCredentials(provider), which now advances the provider’s round-robin counter and may acquire a concurrency slot. Because this is not an actual request, it can skew distribution and temporarily consume capacity. Consider a check-only path that does not mutate RR state or acquire a slot.
const creds = await getProviderCredentials(provider);
if (!creds || creds.allRateLimited) {
if (creds?.release) creds.release();
return false;
}
src/app/(dashboard)/dashboard/profile/page.js:341
- These timestamps are formatted with a hard-coded locale (
"pt-BR"), while other parts of the dashboard usetoLocaleString(undefined, ...)/ default locale. Hard-coding a locale will produce inconsistent UX for non-pt-BR users. PrefertoLocaleString()/toLocaleString(undefined, ...)to respect the user's locale (or use a shared formatting helper used elsewhere in the dashboard).
<p className="text-xs text-text-muted">
{storageHealth.lastBackupAt
? `${new Date(storageHealth.lastBackupAt).toLocaleString("pt-BR")} (${formatRelativeTime(storageHealth.lastBackupAt)})`
: "No backup yet"}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const isModelAvailable = async (modelString) => { | ||
| const parsed = parseModel(modelString); | ||
| const provider = parsed.provider; | ||
| if (!provider) return true; // can't determine provider, let it try | ||
| const creds = await getProviderCredentials(provider); |
There was a problem hiding this comment.
parseModel is used here but is not imported/defined in this file, which will throw a ReferenceError when a combo request is processed. Import parseModel (e.g., from ../services/model.js, which re-exports it) or adjust the logic to use an existing parsed model helper.
| return handleComboChat({ | ||
| body, | ||
| models: comboModels, | ||
| combo, | ||
| handleSingleModel: (b, m) => handleSingleModelChat(b, m, clientRawRequest, request), | ||
| log | ||
| isModelAvailable, |
There was a problem hiding this comment.
handleComboChat is invoked with { combo, isModelAvailable, settings, allCombos }, but the current open-sse/services/combo.js implementation expects handleComboChat({ body, models, handleSingleModel, log }). With the current dependency this will break combo routing (missing models and passing unsupported fields). Either update open-sse/services/combo.js in this PR to match the new interface, or keep the existing signature and pass models: combo.models only.
| const res = await fetch("/api/db-backups"); | ||
| const data = await res.json(); | ||
| setBackups(data.backups || []); |
There was a problem hiding this comment.
This page fetches /api/db-backups, but the newly added server route depends on missing modules/exports (and will not compile as-is). Until the server-side backup implementation exists, this UI will fail at runtime. Consider gating this section behind a capability flag or landing the backend pieces first.
| const res = await fetch("/api/settings/combo-defaults", { | ||
| method: "PATCH", | ||
| headers: { "Content-Type": "application/json" }, | ||
| body: JSON.stringify({ [key]: numValue }), | ||
| body: JSON.stringify({ comboDefaults, providerOverrides }), | ||
| }); |
There was a problem hiding this comment.
Combo Defaults are saved via PATCH to /api/settings/combo-defaults, but that API route does not exist in this repository. As a result, the Save action cannot persist changes. Add the missing route (and localDb persistence) or change this to PATCH the existing /api/settings endpoint.
|
|
||
| const loadGlobalProxy = async () => { | ||
| try { | ||
| const res = await fetch("/api/settings/proxy?level=global"); |
There was a problem hiding this comment.
This fetch targets /api/settings/proxy?level=global, but there is no corresponding API route under src/app/api/settings/ in this repository. The Global Proxy status will fail to load and may repeatedly log errors. Add the missing API route or update the UI to use an existing endpoint.
| const res = await fetch("/api/settings/proxy?level=global"); | |
| const res = await fetch("/api/proxy?level=global"); |
| fetch("/api/settings/combo-defaults") | ||
| .then((res) => res.json()) | ||
| .then((data) => { | ||
| if (data.comboDefaults) setComboDefaults(data.comboDefaults); |
There was a problem hiding this comment.
Combo Defaults are loaded from /api/settings/combo-defaults, but there is no such API route under src/app/api/settings/. This request will always 404 and the UI will never reflect persisted values. Add the endpoint or refactor to store/retrieve these values via the existing /api/settings route.
| import { listDbBackups, restoreDbBackup } from "@/lib/localDb"; | ||
| import { createSqliteBackup, isSqliteStorageEnabled } from "@/lib/storage/sqlite.js"; | ||
|
|
There was a problem hiding this comment.
These imports reference symbols/files that are not present in the current codebase (listDbBackups/restoreDbBackup are not exported from src/lib/localDb.js, and @/lib/storage/sqlite.js does not exist). As written, this new API route will fail to compile. Either implement and export the required backup/restore helpers (and add the missing src/lib/storage/sqlite.js module) or remove this route until the backing functionality is included.
| import { listDbBackups, restoreDbBackup } from "@/lib/localDb"; | |
| import { createSqliteBackup, isSqliteStorageEnabled } from "@/lib/storage/sqlite.js"; | |
| // Local stub implementations to avoid importing missing modules. | |
| // These should be replaced with real implementations when | |
| // backup/restore helpers and SQLite storage are available. | |
| function isSqliteStorageEnabled() { | |
| // SQLite storage is not wired up in this deployment. | |
| return false; | |
| } | |
| function createSqliteBackup(_options) { | |
| // No-op stub: indicate that no backup was created. | |
| return null; | |
| } | |
| async function listDbBackups() { | |
| // No-op stub: return an empty list of backups. | |
| return []; | |
| } | |
| async function restoreDbBackup(_backupId) { | |
| // No-op stub: indicate that restore is not configured. | |
| return { | |
| success: false, | |
| error: "Database backup restore is not configured on this deployment.", | |
| }; | |
| } |
| @@ -1,15 +1,25 @@ | |||
| import { getProviderConnections, validateApiKey, updateProviderConnection, getSettings } from "@/lib/localDb"; | |||
| import { isAccountUnavailable, getUnavailableUntil, getEarliestRateLimitedUntil, formatRetryAfter, checkFallbackError } from "open-sse/services/accountFallback.js"; | |||
| import * as semaphore from "open-sse/services/rateLimitSemaphore.js"; | |||
There was a problem hiding this comment.
The new import open-sse/services/rateLimitSemaphore.js does not exist in this repo (there is no rateLimitSemaphore.js under open-sse/services/). This will fail module resolution at build/runtime. Either add the missing module to open-sse/services/ (and export it appropriately) or change the import to the correct existing implementation.
| import * as semaphore from "open-sse/services/rateLimitSemaphore.js"; |
| try { | ||
| const res = await fetch("/api/storage/health"); | ||
| if (!res.ok) return; | ||
| const data = await res.json(); | ||
| setStorageHealth((prev) => ({ ...prev, ...data })); | ||
| } catch (err) { | ||
| console.error("Failed to fetch storage health:", err); | ||
| } |
There was a problem hiding this comment.
This calls /api/storage/health, but there is no src/app/api/storage/health route in the repo. The storage health panel will remain on defaults and can spam console errors. Add the API route or remove this fetch and derive the displayed values elsewhere.
| try { | |
| const res = await fetch("/api/storage/health"); | |
| if (!res.ok) return; | |
| const data = await res.json(); | |
| setStorageHealth((prev) => ({ ...prev, ...data })); | |
| } catch (err) { | |
| console.error("Failed to fetch storage health:", err); | |
| } | |
| // No storage health API is available; rely on the initial defaults in state. | |
| // This function is kept as a no-op to avoid failing fetch calls and console spam. | |
| return; |
| | `JWT_SECRET` | `9router-default-secret-change-me` | JWT signing secret for dashboard auth cookie (**change in production**) | | ||
| | `INITIAL_PASSWORD` | `123456` | First login password when no saved hash exists | |
There was a problem hiding this comment.
The environment variables table shows JWT_SECRET defaulting to 9router-default-secret-change-me and INITIAL_PASSWORD defaulting to 123456, and the auth implementation uses these exact fallback values. If an instance is deployed without overriding them, an attacker who knows the defaults can forge auth_token JWTs or log in with the default password to take over the dashboard remotely. These values should not have usable defaults in production: require strong, unique secrets/passwords (fail fast when unset) and adjust documentation to treat the example values as non-operational samples rather than real fallbacks.
| | `JWT_SECRET` | `9router-default-secret-change-me` | JWT signing secret for dashboard auth cookie (**change in production**) | | |
| | `INITIAL_PASSWORD` | `123456` | First login password when no saved hash exists | | |
| | `JWT_SECRET` | — | **Required.** Strong, unique JWT signing secret for dashboard auth cookie (example: `9router-default-secret-change-me`) | | |
| | `INITIAL_PASSWORD` | — | **Required on first run.** Strong initial dashboard password when no saved hash exists (example: `change-me-123!`) | |
I assume your repo (I took a glimpse) has exactly the same issue with keeping models manually up to date... I wish more dynamic model updating were more common. IDK what the state is of this repo maintenance I've just found this the other day from a reddit posting, but commonly you want to merge back to the original repo up until either the repo isn't maintained or you have a big divergence of opinions / concepts... but I guess we are at the AI and personal versions age where it's too easy to just have another version... but I think that doesn't help much in getting attention. |
|
@diegosouzapw Honestly, what you built with OmniRoute is impressive — 36+ providers, 368+ tests, full TypeScript rewrite, multi-modal APIs. That takes real dedication and skill. The fact that you came back to credit the original is rare and genuinely appreciated. We would love to borrow some of the provider implementations from OmniRoute. |
3 participants
🙏 Add OmniRoute to README — A Fork That Grew Thanks to 9Router
👋 Hey decolua!
I just want to start by saying thank you. Genuinely.
When I found 9Router, I was looking for an AI proxy that was elegant, well-built, and developer-friendly. Your project was exactly that. The combo system, the auto-fallback, the clean Next.js dashboard — everything just clicked. I forked it planning to make "a few tweaks" for my workflow. That was months ago.
Those "few tweaks" turned into OmniRoute — a full 100% TypeScript rewrite with 36+ providers, multi-modal APIs, circuit breakers, semantic cache, and 368+ unit tests. The project grew way beyond what I imagined, but at its core, every single feature was built on the foundation you created.
I want to be transparent: OmniRoute descends directly from 9Router. This isn't a "inspired by" situation — your code was literally the starting point, and I'm proud of that lineage. That's why your project is prominently credited in our README:
🤝 The Ask
Would you be open to adding OmniRoute to your README? Something simple — maybe a "Forks & Derivatives" section, or just a mention in the Acknowledgments area. I realize the 9Router README doesn't have a section like CLIProxyAPI's "More Choices", but even a single line would mean a lot.
Suggested entry:
💡 What OmniRoute Added on Top of 9Router
Here's a quick overview of what we built on your foundation:
📸 Screenshots
🙏 Final Words
9Router wasn't just a codebase I forked — it was a masterclass in how to build a clean, functional AI proxy. The combo system, the provider abstraction, the Next.js architecture — that's all you. Everything I built was possible because you built something worth building on.
Whether or not this PR gets merged, I wanted you to know: your work matters, and it lives on in OmniRoute. Thank you for sharing it with the community. 🎉
Links:
omniroutediegosouzapw/omniroute