feat(llms-txt): add llms.txt generation for npm packages #1382
Conversation
Add AgentFile and LlmsTxtResult interfaces for llms.txt generation and export from the shared types barrel.
Add discoverAgentFiles, fetchAgentFiles, generateLlmsTxt, and handleLlmsTxt orchestrator for llms.txt generation from npm packages.
Serve llms.txt at /api/registry/llms-txt/[...pkg] following existing registry API patterns with cached event handler and SWR.
Cover discoverAgentFiles, fetchAgentFiles, and generateLlmsTxt including root files, directory scanning, graceful failures, scoped packages, and full/minimal output generation.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
📝 WalkthroughWalkthroughThis pull request introduces support for generating llms.txt files for npm packages on npmx.dev. It adds a new API route that serves dynamically generated llms.txt Markdown documents for requested packages. The implementation discovers well-known agent instruction files (CLAUDE.md, AGENTS.md, .cursor/rules, .github/copilot-instructions.md, etc.) within packages, fetches their contents from jsDelivr CDN, and assembles them into a structured llms.txt format alongside package metadata and README content. The route includes input validation, error handling, and HTTP caching with a one-hour max age. Supporting utilities and TypeScript type definitions are provided for content generation and data structure definitions. Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Use Nitro server routes at /package/.../llms.txt instead of an API route with middleware rewriting. Single handler re-exported across four route files for unscoped, scoped, and versioned URL patterns.
1 participant
Summary
/package/.../llms.txtfor any npm packageCloses #1378
URL patterns
/package/nuxt/llms.txt— latest version ✅/package/@nuxt/kit/llms.txt— scoped, latest ✅/package/nuxt/v/3.12.0/llms.txt— specific version/package/@nuxt/kit/v/1.0.0/llms.txt— scoped, specific versionFiles
server/routes/package/[name]/llms.txt.get.ts— main cached handlerserver/routes/package/[org]/[name]/llms.txt.get.ts— re-exports main handler for scoped packagesserver/routes/package/[name]/v/[version]/llms.txt.get.ts— re-exports for versionedserver/routes/package/[org]/[name]/v/[version]/llms.txt.get.ts— re-exports for scoped + versionedserver/utils/llms-txt.ts— core logic (discover agent files, fetch, generate markdown)shared/types/llms-txt.ts— type definitionsnuxt.config.ts— ISR caching via/package/**/llms.txtglobTest plan
pnpm test:unit— 16 new tests covering discoverAgentFiles, fetchAgentFiles, generateLlmsTxtpnpm dev→ verified unscoped and scoped (latest) URL patterns returntext/markdown/v/:version/llms.txt)