Refactor: Use MCP registry API as source of truth

[aaronpowell]

• Replace static github-mcp-registry.json with live API calls
• Fetch from https://api.mcp.github.com/v0.1/servers/ with pagination support
• Extract displayName from GitHub metadata for better matching
• Implement smart matching logic:
  • Case-insensitive comparison
  • Match against both displayName and full name
  • Strip common suffixes like -mcp-server for pattern matching
• Make build process async to support API calls
• Cache registry data to only hit API once per build run
• Remove obsolete github-mcp-registry.json file

Benefits:

• No more manual updates to registry data
• Always uses latest MCP registry information
• Improved server name matching resilience
• Successfully loads all 54 servers from registry

Pull Request Checklist

• I have read and followed the CONTRIBUTING.md guidelines.
• My contribution adds a new instruction, prompt, or chat mode file in the correct directory.
• The file follows the required naming convention.
• The content is clearly structured and follows the example format.
• I have tested my instructions, prompt, or chat mode with GitHub Copilot.
• I have run npm start and verified that README.md is up to date.

---

Description

---

Type of Contribution

• New instruction file.
• New prompt file.
• New chat mode file.
• New collection file.
• Update to existing instruction, prompt, chat mode, or collection.
• Other (please specify):

---

Additional Notes

---

By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

[Copilot]

Pull request overview

This PR refactors the MCP registry data source from a static JSON file to live API calls, enabling automatic updates and eliminating manual maintenance. The implementation introduces async/await patterns, cursor-based pagination, and smart matching logic to handle server name variations.

Key Changes:

• Converted loadMcpRegistryNames() to async function that fetches from https://api.mcp.github.com/v0.1/servers/ with pagination
• Implemented smart matching logic supporting displayName, fullName, and partial name matching with suffix stripping
• Wrapped main execution in async function to support API calls during build process

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 6 comments.

File	Description
eng/update-readme.mjs	Core refactor: async API fetching with pagination, smart matching logic, and parameter threading throughout the build pipeline
docs/README.agents.md	Generated documentation reflecting updated MCP server GitHub URLs from live registry data
collections/partners.md	Generated collection file with updated MCP server GitHub URLs

[Copilot]

The JSDoc comment states "Extracts server names from: data[].server.name" but the actual code accesses json?.servers (not data) and then entry?.server?.name. The documentation should accurately reflect the actual API response structure being accessed to avoid confusion for future maintainers.

Suggested change

	* - Extracts server names from: data[].server.name
	* - Extracts server names from: servers[].server.name

[Copilot]

This line and subsequent lines are missing proper indentation. The code starting from line 917 (collectionsReadme declaration) should be indented at the same level as the previous statements (lines 894-915) since they are all part of the same try block that starts at line 876. This inconsistent indentation reduces code readability.

See below for a potential fix:

    // Generate collections README
    const collectionsReadme = buildCategoryReadme(
      generateCollectionsSection,
      COLLECTIONS_DIR,
      collectionsHeader,
      TEMPLATES.collectionsUsage,
      registryNames
    );

    // Ensure docs directory exists for category outputs
    if (!fs.existsSync(DOCS_DIR)) {
      fs.mkdirSync(DOCS_DIR, { recursive: true });
    }

    // Write category outputs into docs folder
    writeFileIfChanged(
      path.join(DOCS_DIR, "README.instructions.md"),
      instructionsReadme
    );
    writeFileIfChanged(path.join(DOCS_DIR, "README.prompts.md"), promptsReadme);
    writeFileIfChanged(path.join(DOCS_DIR, "README.agents.md"), agentsReadme);
    writeFileIfChanged(
      path.join(DOCS_DIR, "README.collections.md"),
      collectionsReadme
    );

[Copilot]

Consider adding response body logging or context to the error message when JSON parsing fails. Currently, if response.json() throws an error (e.g., due to malformed JSON), the error message will only contain the JSON parsing error without information about which API call failed. This makes debugging production issues more difficult. Consider wrapping this in a try-catch to provide better error context.

Suggested change

	const json = await response.json();
	let json;
	try {
	json = await response.json();
	} catch (parseErr) {
	// Try to get the response body as text for debugging
	let responseText;
	try {
	responseText = await response.text();
	} catch (textErr) {
	responseText = `<failed to read response body: ${textErr}>`;
	}
	// Limit the response body snippet to 500 chars to avoid log spam
	const snippet = responseText && responseText.length > 500
	? responseText.slice(0, 500) + '...'
	: responseText;
	throw new Error(
	`Failed to parse JSON from ${url}. Error: ${parseErr.message}. Response body: ${snippet}`
	);
	}

[Copilot]

The matching logic has redundant conditions. Line 435 checks if entry.displayName === serverNameLower, and line 451 repeats the exact same check. Since line 435 is inside the same conditional block and would already return true if this condition matched, the check on line 451 is unreachable and serves no purpose. Consider removing line 451 or restructuring the logic to eliminate this redundancy.

Suggested change

return entry.displayName === serverNameLower;

[Copilot]

The fetch call to the MCP registry API lacks timeout configuration. If the API becomes unresponsive, the build process could hang indefinitely. Consider adding a timeout using AbortController or a timeout option to fail fast and avoid blocking the build process. This is especially important since this is a build-time dependency that affects the entire README generation process.

Suggested change

	const response = await fetch(url);
	// Add timeout to fetch using AbortController
	const controller = new AbortController();
	const timeout = setTimeout(() => controller.abort(), 10000); // 10 seconds
	let response;
	try {
	response = await fetch(url, { signal: controller.signal });
	} finally {
	clearTimeout(timeout);
	}

[Copilot]

The pagination loop with a do-while structure could potentially run indefinitely if the API returns malformed pagination metadata (e.g., continuously returning the same cursor). Consider adding a safety limit (e.g., maximum number of pages or iterations) to prevent infinite loops in case of API bugs or unexpected behavior. This would make the build process more resilient to API issues.