feat: add Astrai intelligent router to summarization fallback chain#115
feat: add Astrai intelligent router to summarization fallback chain#115beee003 wants to merge 1 commit intokoala73:mainfrom
Conversation
…chain Adds Astrai (AI inference router) as a provider in the summarization fallback chain: Groq → Astrai → OpenRouter → Browser T5. Astrai auto-selects the optimal model/provider per request based on cost, latency, and task complexity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@beemaya003 is attempting to deploy a commit to the eliehabib projects Team on Vercel. A member of the Team first needs to authorize it. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22b5563c61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const data = await response.json(); | ||
| const provider = data.cached ? 'cache' : 'astrai'; | ||
| console.log(`[Summarization] ${provider === 'cache' ? 'Redis cache hit' : 'Astrai success'}:`, data.model); | ||
| return { | ||
| summary: data.summary, |
There was a problem hiding this comment.
api/astrai-summarize.js returns HTTP 200 with {fallback: true, skipped: true, summary: null} when ASTRAI_API_KEY is not configured (see the early return around lines 90-95), but this branch accepts any 2xx as success and returns data.summary without checking fallback/summary. That makes generateSummary stop at Astrai with a null summary when Groq fails (e.g., rate limiting) and Astrai is unconfigured, instead of continuing to OpenRouter/Browser fallback.
Useful? React with 👍 / 👎.
| if (response.ok) { | ||
| const data = await response.json(); | ||
| return data.summary; |
There was a problem hiding this comment.
This translation path returns data.summary for any HTTP-200 Astrai response, but the Astrai endpoint uses HTTP 200 for "skipped" fallback responses when the key is missing. In that case data.summary is null, and returning it exits translateText before the OpenRouter step, so translations fail whenever Groq fails and Astrai is enabled but not configured.
Useful? React with 👍 / 👎.
koala73
added
enhancement
4 participants
Summary
strategy: "cheapest"since summarization is lightweight — routes to the most cost-effective model that meets quality requirementsmodel: "auto"so Astrai picks the best model for each request (e.g., Groq Llama for simple summaries, Claude for complex analysis)Why this fits WorldMonitor
WorldMonitor already has a smart fallback chain (Groq → OpenRouter → Browser T5). Astrai slots in naturally as a second cloud fallback that routes across all providers (OpenAI, Anthropic, Groq, DeepInfra, etc.) from a single API key. When Groq is rate-limited (14.4K/day), Astrai can find the cheapest alternative across all providers instead of falling through to OpenRouter's 50/day free tier.
Changes
api/astrai-summarize.jssrc/services/summarization.tstryAstrai()+ wire into all fallback paths (normal, beta warm, beta cold, translation)src/services/runtime-config.tsASTRAI_API_KEYsecret,aiAstraifeature toggle, feature definitionConfiguration
Set
ASTRAI_API_KEYas a Vercel environment variable (or desktop keychain secret). The feature toggleaiAstraiis enabled by default and appears in the Runtime Config panel.# Vercel vercel env add ASTRAI_API_KEYDesign decisions
summary:v3:...cache format as Groq/OpenRouter, so a cached response from any provider serves all threestrategy: "cheapest"— Summarization tasks are lightweight (150 max tokens), so cost optimization makes sense over quality optimizationASTRAI_API_KEYis not set, the endpoint returns{ fallback: true, skipped: true }and the chain continues to OpenRouterTest plan
ASTRAI_API_KEYis not setASTRAI_API_KEYset — Astrai should handle requests when Groq failsaiAstraitoggle appears in Runtime Config panel and can be disabled🤖 Generated with Claude Code