Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Could you add a link from it in the footer? |
|
@atinux in which section? |
|
Explore > AI Evals Another suggestion is to remove Modules in the footer as already in the header now and have:
|
📝 WalkthroughWalkthroughThis PR introduces a new agent evaluation results dashboard feature. Changes include adding a navigation link to AI Evals (replacing Modules in the footer), creating a new evals.vue page that displays agent performance metrics in a table with filtering and expandable row functionality, configuring a new content collection in content.config.ts, and adding supporting static data files (YAML metadata and JSON benchmark results containing experiments and per-model evaluation outcomes). Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
content.config.ts (1)
425-433:githubUrlshould use.url()validation for consistency.Other URL fields in this config (e.g.,
demo,purchase,website) usez.string().url(). Apply the same validation here.♻️ Suggested fix
evals: defineCollection({ type: 'data', source: 'evals.yml', schema: z.object({ title: z.string(), description: z.string(), - githubUrl: z.string() + githubUrl: z.string().url() }) })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content.config.ts` around lines 425 - 433, The evals collection schema currently validates githubUrl with z.string(); update the schema in the defineCollection call for evals to use z.string().url() for githubUrl so it matches other URL fields (demo, purchase, website) and enforces proper URL validation; locate the evals defineCollection and replace the githubUrl schema entry accordingly.app/pages/evals.vue (2)
50-53: Fetching own static asset via full site URL is suboptimal for SSR.
$fetch(joinURL(url, '/agent-results.json'))makes an external HTTP request to the app's own origin during SSR. Since this is a file inpublic/, you can simplify to$fetch('/agent-results.json')— Nuxt's$fetchhandles this internally during SSR without a network round-trip.♻️ Suggested fix
const [{ data: page }, { data: rawData }] = await Promise.all([ useAsyncData('evals', () => queryCollection('evals').first()), - useAsyncData('agent-results', () => $fetch<AgentResultsData>(joinURL(url, '/agent-results.json'))) + useAsyncData('agent-results', () => $fetch<AgentResultsData>('/agent-results.json')) ])This also removes the need for the
useSiteConfig()call on line 48 and thejoinURLimport on line 4 (if unused elsewhere).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/pages/evals.vue` around lines 50 - 53, The $fetch call is making an external request via joinURL(url, '/agent-results.json') during SSR; change the useAsyncData invocation that fetches agent-results to call $fetch('/agent-results.json') instead (e.g., inside the second useAsyncData for 'agent-results') so Nuxt serves the public file without a network round-trip, and remove the now-unused useSiteConfig() call and joinURL import if they are no longer referenced.
122-134: Model avatar mapping is brittle and will need manual updates for every new model.This works for the current dataset but consider externalizing this mapping (e.g., into the JSON data or a config file) so new models don't require code changes.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/pages/evals.vue` around lines 122 - 134, getModelAvatar currently hardcodes string checks for model names making it brittle; refactor by moving the model-to-avatar mapping out of the function into a configurable data source (e.g., a JSON config or a constant imported from a config module) and update getModelAvatar to look up the lowercased model key in that mapping with a sensible default fallback; specifically, create a mapping object (or load JSON) keyed by normalized model identifiers, import/use it inside getModelAvatar, replace the series of if checks with a single map lookup, and ensure the function still returns undefined or a default avatar when no match is found.app/composables/useNavigation.ts (1)
136-145: Inconsistent URL format: relative vs absolute paths in the same footer section.
TemplatesandShowcaseuse absolute URLs (https://nuxt.com/...) while the newAI Evalsentry uses a relative path (/evals). Consider using consistent path formats within the same section.♻️ Suggested fix
}, { label: 'AI Evals', - to: '/evals' + to: 'https://nuxt.com/evals' }]Or alternatively, make Templates and Showcase relative too.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/composables/useNavigation.ts` around lines 136 - 145, The footer navigation "children" array in useNavigation.ts mixes absolute URLs ('Templates' and 'Showcase' use https://nuxt.com/...) with a relative path for 'AI Evals' ('/evals'), causing inconsistent link behavior; update the entry for 'AI Evals' (the object with label 'AI Evals' inside the children array) to use the same URL format as the other items (e.g., change to 'https://nuxt.com/evals') or alternatively make the 'Templates' and 'Showcase' entries relative to match '/evals'—apply the change in the children array where these objects are defined.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@app/pages/evals.vue`:
- Around line 85-99: The computed allResults can divide by zero when evals is
empty; update the loop inside the allResults computed to guard against
evals.length === 0 by computing successRate only when evals.length > 0 (e.g.,
set successRate to 0 or a safe fallback), i.e. compute successes from evals, set
const denom = evals.length, then use denom ? Math.round((successes/denom)*100) :
0 when building the ModelRow object (referencing allResults, ModelRow, evals,
successes, and successRate).
---
Nitpick comments:
In `@app/composables/useNavigation.ts`:
- Around line 136-145: The footer navigation "children" array in
useNavigation.ts mixes absolute URLs ('Templates' and 'Showcase' use
https://nuxt.com/...) with a relative path for 'AI Evals' ('/evals'), causing
inconsistent link behavior; update the entry for 'AI Evals' (the object with
label 'AI Evals' inside the children array) to use the same URL format as the
other items (e.g., change to 'https://nuxt.com/evals') or alternatively make the
'Templates' and 'Showcase' entries relative to match '/evals'—apply the change
in the children array where these objects are defined.
In `@app/pages/evals.vue`:
- Around line 50-53: The $fetch call is making an external request via
joinURL(url, '/agent-results.json') during SSR; change the useAsyncData
invocation that fetches agent-results to call $fetch('/agent-results.json')
instead (e.g., inside the second useAsyncData for 'agent-results') so Nuxt
serves the public file without a network round-trip, and remove the now-unused
useSiteConfig() call and joinURL import if they are no longer referenced.
- Around line 122-134: getModelAvatar currently hardcodes string checks for
model names making it brittle; refactor by moving the model-to-avatar mapping
out of the function into a configurable data source (e.g., a JSON config or a
constant imported from a config module) and update getModelAvatar to look up the
lowercased model key in that mapping with a sensible default fallback;
specifically, create a mapping object (or load JSON) keyed by normalized model
identifiers, import/use it inside getModelAvatar, replace the series of if
checks with a single map lookup, and ensure the function still returns undefined
or a default avatar when no match is found.
In `@content.config.ts`:
- Around line 425-433: The evals collection schema currently validates githubUrl
with z.string(); update the schema in the defineCollection call for evals to use
z.string().url() for githubUrl so it matches other URL fields (demo, purchase,
website) and enforces proper URL validation; locate the evals defineCollection
and replace the githubUrl schema entry accordingly.
2 participants
🔗 Linked issue
❓ Type of change
📚 Description