feat(gitea): add edge caching for tickets API to reduce Gitea traffic#629
feat(gitea): add edge caching for tickets API to reduce Gitea traffic#629dadofsambonzuki wants to merge 4 commits intomainfrom
Conversation
Replace ineffective in-memory cache with 24-hour edge cache at Netlify CDN. All ticket pages now fetch from a single /api/tickets endpoint with proper pagination support for 1000+ issues. 🤖 Generated with [opencode](https://opencode.ai)
✅ Deploy Preview for btcmap ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
||||||||||||||||||||||||||
This comment was marked as resolved.
This comment was marked as resolved.
Gitea's API may cap the limit parameter at 50, causing pagination to break when requesting 100. Reduced to 50 to ensure proper pagination through all open issues. 🤖 Generated with [opencode](https://opencode.ai)
The edge cache captured the initial broken response with only 50 issues. Adding ?v=2 creates a new cache key to bypass the stale cached response. 🤖 Generated with [opencode](https://opencode.ai)
… cache" This reverts commit 1ea6702.
| axiosRetry(axios, { retries: 3, retryDelay: axiosRetry.exponentialDelay }); | ||
|
|
||
| export const load: PageServerLoad = async ({ params }) => { | ||
| type TicketsResponse = { |
There was a problem hiding this comment.
This is already defined above, can we move this to types or gitea.ts?
| error?: string; | ||
| }; | ||
|
|
||
| function filterIssuesByLabel(issues: GiteaIssue[], labelName: string): GiteaIssue[] { |
There was a problem hiding this comment.
This and below also duplicates above. Maybe the AI has an idea how to DRY this reasonabale?
| import type { PageServerLoad } from './$types'; | ||
|
|
||
| export const load: PageServerLoad = async () => { | ||
| type TicketsResponse = { |
There was a problem hiding this comment.
Pull request overview
This PR replaces the ineffective in-memory caching (incompatible with serverless architecture) with a centralized edge-cached API endpoint at /api/tickets. The changes reduce Gitea API traffic from potentially hundreds of calls per day to approximately one call per day through a 24-hour CDN cache.
- Adds new
/api/ticketsendpoint with proper pagination to fetch all 1000+ open Gitea issues - Removes in-memory cache from
gitea.tsmodule - Updates tickets, country, and community pages to fetch from the cached endpoint with client-side label filtering
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| src/routes/api/tickets/+server.ts | New API endpoint with 24-hour edge caching and pagination support for fetching all Gitea issues |
| src/lib/gitea.ts | Removes in-memory caching logic and deleted getIssues() function; retains label management and issue creation |
| src/routes/tickets/+page.server.ts | Updates to fetch from cached /api/tickets endpoint instead of calling getIssues() directly |
| src/routes/country/[area]/[section]/+page.server.ts | Fetches from cached endpoint and adds client-side label filtering for area-specific tickets |
| src/routes/community/[area]/[section]/+page.server.ts | Fetches from cached endpoint and adds client-side label filtering for area-specific tickets |
| export const load: PageServerLoad = async ({ fetch }) => { | ||
| try { | ||
| const { issues, totalCount } = await getIssues(); | ||
| const response = await fetch('/api/tickets'); |
There was a problem hiding this comment.
The fetch call doesn't check the response status before calling json(). If the API returns a non-200 status (e.g., 500 error), this will still attempt to parse JSON and may not properly propagate the error. Consider checking response.ok before parsing.
| const response = await fetch('/api/tickets'); | |
| const response = await fetch('/api/tickets'); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch tickets: ${response.status} ${response.statusText}`); | |
| } |
| type TicketsResponse = { | ||
| issues: GiteaIssue[]; | ||
| totalCount: number; | ||
| error?: string; | ||
| }; |
There was a problem hiding this comment.
The TicketsResponse type is duplicated across three files (tickets page, country page, and community page). This violates the DRY principle and makes maintenance harder. Consider moving this type to a shared location like src/lib/types.ts where GiteaIssue is already defined.
| const ticketsResponse = await fetch('/api/tickets'); | ||
| const ticketsData: TicketsResponse = await ticketsResponse.json(); |
There was a problem hiding this comment.
The fetch call doesn't check the response status before calling json(). If the API returns a non-200 status (e.g., 500 error), this will still attempt to parse JSON and may not properly propagate the error. Consider checking response.ok before parsing.
| type TicketsResponse = { | ||
| issues: GiteaIssue[]; | ||
| totalCount: number; | ||
| error?: string; | ||
| }; |
There was a problem hiding this comment.
The TicketsResponse type is duplicated across multiple files. This violates the DRY principle and makes maintenance harder. Consider moving this type to a shared location like src/lib/types.ts where GiteaIssue is already defined.
| type TicketsResponse = { | ||
| issues: GiteaIssue[]; | ||
| totalCount: number; | ||
| error?: string; | ||
| }; |
There was a problem hiding this comment.
The TicketsResponse type is duplicated across multiple files. This violates the DRY principle and makes maintenance harder. Consider moving this type to a shared location like src/lib/types.ts where GiteaIssue is already defined.
| import axios from 'axios'; | ||
| import { env } from '$env/dynamic/private'; | ||
|
|
||
| import type { RequestHandler } from './$types'; | ||
| import type { GiteaIssue } from '$lib/types'; | ||
|
|
There was a problem hiding this comment.
Missing axios-retry configuration for resilience. Other API endpoints in this codebase use axiosRetry with exponential backoff for better reliability. Consider adding similar retry logic here to handle transient Gitea API failures, especially since this endpoint is cached for 24 hours.
| import axios from 'axios'; | |
| import { env } from '$env/dynamic/private'; | |
| import type { RequestHandler } from './$types'; | |
| import type { GiteaIssue } from '$lib/types'; | |
| import axios from 'axios'; | |
| import axiosRetry from 'axios-retry'; | |
| import { env } from '$env/dynamic/private'; | |
| import type { RequestHandler } from './$types'; | |
| import type { GiteaIssue } from '$lib/types'; | |
| axiosRetry(axios, { | |
| retries: 3, | |
| retryDelay: axiosRetry.exponentialDelay, | |
| retryCondition: (error) => | |
| axiosRetry.isNetworkError(error) || axiosRetry.isRetryableError(error) | |
| }); |
| function filterIssuesByLabel(issues: GiteaIssue[], labelName: string): GiteaIssue[] { | ||
| return issues.filter((issue) => | ||
| issue.labels.some((label) => label.name.toLowerCase() === labelName.toLowerCase()) | ||
| ); | ||
| } |
There was a problem hiding this comment.
The filterIssuesByLabel function is duplicated in both country and community page server files. This duplicated logic should be extracted to a shared utility module to avoid maintenance issues and ensure consistent behavior.
| const ticketsResponse = await fetch('/api/tickets'); | ||
| const ticketsData: TicketsResponse = await ticketsResponse.json(); |
There was a problem hiding this comment.
The fetch call doesn't check the response status before calling json(). If the API returns a non-200 status (e.g., 500 error), this will still attempt to parse JSON and may not properly propagate the error. Consider checking response.ok before parsing.
| while (true) { | ||
| const response = await axios.get( | ||
| `${env.GITEA_API_URL}/api/v1/repos/teambtcmap/btcmap-data/issues?state=open&limit=${limit}&page=${page}`, | ||
| { headers } | ||
| ); | ||
|
|
||
| const issues = response.data.map((issue: GiteaIssue) => ({ | ||
| id: issue.id, | ||
| number: issue.number, | ||
| title: issue.title, | ||
| created_at: issue.created_at, | ||
| html_url: issue.html_url, | ||
| labels: issue.labels, | ||
| user: { | ||
| login: issue.user.login, | ||
| avatar_url: issue.user.avatar_url, | ||
| html_url: issue.user.html_url | ||
| }, | ||
| comments: issue.comments, | ||
| assignees: issue.assignees | ||
| })); | ||
|
|
||
| allIssues.push(...issues); | ||
|
|
||
| // If we got fewer than requested, we've reached the end | ||
| if (response.data.length < limit) break; | ||
| page++; | ||
| } |
There was a problem hiding this comment.
The pagination loop lacks safeguards against infinite loops. If the API response is malformed or if the limit check fails, this could loop indefinitely. Consider adding a maximum page limit (e.g., 100 pages) to prevent potential runaway execution in serverless environments.
| return { | ||
| tickets: issues, | ||
| totalTickets: totalCount | ||
| tickets: data.issues, | ||
| totalTickets: data.totalCount | ||
| }; |
There was a problem hiding this comment.
The error field from the API response is not being checked or propagated. When the API returns a 500 status with an error message in the response body, this code will successfully parse the JSON and return empty arrays without surfacing the actual error message to the user. Consider checking data.error and returning it if present.
User description
Summary
/api/ticketsendpoint with 24-hour edge caching at Netlify CDN/tickets,/country/*/maintain, and/community/*/maintainpages to use the cached endpointProblem
The previous in-memory cache was incompatible with Netlify's serverless architecture:
This resulted in potentially hundreds of Gitea API calls per day.
Solution
Single cached API endpoint at
/api/tickets:Cache-Control: public, max-age=86400, stale-while-revalidate=86400(24hr cache)Expected Impact
See
GITEA_CACHING_ANALYSIS.mdfor full analysis.PR Type
Enhancement
Description
Replace in-memory cache with 24-hour edge cache at Netlify CDN
Add new
/api/ticketsendpoint with proper pagination supportUpdate ticket pages to fetch from centralized cached endpoint
Remove client-side cache invalidation logic from issue creation
Diagram Walkthrough
File Walkthrough
gitea.ts
Remove in-memory caching logic from gitea modulesrc/lib/gitea.ts
syncIssuesFromGitea()andgetIssues()functionscreateIssueWithLabels()+page.server.ts
Update tickets page to use cached API endpointsrc/routes/tickets/+page.server.ts
getIssues()call with fetch to/api/ticketsendpointTicketsResponsetype definition for API response+page.server.ts
Update country pages to use cached tickets endpointsrc/routes/country/[area]/[section]/+page.server.ts
getIssues()with fetch to/api/ticketsendpointfilterIssuesByLabel()helper function for client-side filteringTicketsResponsetype definition for API response+page.server.ts
Update community pages to use cached tickets endpointsrc/routes/community/[area]/[section]/+page.server.ts
getIssues()with fetch to/api/ticketsendpointfilterIssuesByLabel()helper function for client-side filteringTicketsResponsetype definition for API response+server.ts
New edge-cached tickets API endpoint with paginationsrc/routes/api/tickets/+server.ts