Conversation
Captures all 23 entities, 20 behavioural rules, 12 surface contracts, and 7 config constants extracted from the existing codebase. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add DiscoveredVacancy canonical type, DataSourceConnector interface, EuresAPI external entity, ConnectorDispatch rule, and extend Automation.jobBoard enum with eures support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce DataSourceConnector interface, DiscoveredVacancy canonical type, and ConnectorRegistry. Refactor JSearch into a connector implementation with ACL translator. Runner now resolves connectors from the registry using automation.jobBoard. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement EURES connector using types generated from the EURES OpenAPI spec via openapi-typescript. ACL translator maps EURES vacancies to canonical DiscoveredVacancy type. Registered in connector registry. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add optional connectorParams field to Prisma schema, TypeScript interface, and Zod schema. Runner passes parsed params to connector search method for provider-specific configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add EURES as selectable data source in the automation creation wizard. Location field description adapts to show NUTS code hint when EURES is selected. Review step displays proper connector display names. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add async autocomplete combobox that queries the EURES ESCO occupation API when the job board is set to EURES. Includes a Next.js proxy route for CORS avoidance, debounced search with vacancy frequency display, and graceful degradation on upstream errors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace blanket form.watch() with form.watch("jobBoard") and
form.watch("connectorParams") for conditional rendering, and
form.getValues() for on-demand reads in step validation and review.
Eliminates unnecessary re-renders when typing in unrelated fields.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Align TriggerRun surface action with rule trigger name, add TriggerRun to AutomationListSurface, and add when-guard showing rate limit constraint on AutomationDetailSurface. Manual runs allowed regardless of automation status (active or paused). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace single-page fetch (10 results) with pagination loop fetching 50 results per page until all pages are exhausted. Breaks on empty response or when accumulated count reaches numberRecords. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract retry (exponential backoff), circuit breaker (5 consecutive failures, 30s half-open), cooperative timeout (15s), and bulkhead (5 concurrent, 10 queued) into shared resilience module. Generic resilientFetch<T> wrapper reusable across all EURES API endpoints. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix ConnectorError to use variant-specific fields (reason for blocked, message for network/parse, retryAfter for rate_limited). Update EuresAPI baseUrl and endpoint paths to match actual API URLs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add getDetails to EURES connector for fetching full vacancy detail
via /jv/id/{id}. Narrow employmentType from string to union type
across both connectors. Include city names in EURES location output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After dedup/cap, fetch EURES detail data (applicationDeadline, applicationInstructions) for up to 10 jobs via getDetails. Include enriched fields in LLM prompt for improved match quality. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevent 400/429 errors during pagination and detail enrichment by pacing requests with a token bucket (capacity 3, 500ms refill ≈ 2 req/s). Rate limiting applies before the cockatiel policy so retries also respect the limit, preventing retry storms. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The EURES API returns null for translations, employer, locationMap, and positionScheduleCodes despite the OpenAPI schema declaring them as required. Add defensive optional chaining and nullish coalescing to prevent runtime crashes during vacancy translation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds nullable TEXT column to Automation table for storing connector-specific parameters (e.g. EURES ESCO occupation codes). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolve conflicts in scraper layer keeping connector abstraction pattern: - jsearch/index.ts: keep createJSearchConnector, fix apiKey→RAPIDAPI_KEY - mapper.ts: keep vacancy field names, adopt userId in getOrCreateJobSource - runner.ts: keep connectorRegistry.create over direct searchJSearchJobs call Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Here's a quick walkthrough of what this PR does: Connector abstraction layer — The scraper was previously hardcoded to JSearch/RapidAPI. I introduced a EURES integration — Added the EU's public EURES job portal as a second data source. Types are generated from their OpenAPI spec, and an ACL translator maps their vacancy format to our canonical Resilience — The EURES API can be flaky, so the connector uses cockatiel for retry with exponential backoff, a circuit breaker, timeouts, and a bulkhead for concurrency control. There's also a token bucket rate limiter to stay within their rate limits during pagination and detail enrichment. UI changes — EURES shows up as a selectable data source in the automation wizard. When selected, the keywords field switches to an ESCO occupation autocomplete (with vacancy frequency counts). Also fixed some unnecessary re-renders from Allium specs — Added domain specifications that document the bounded contexts, entities, and behavioural rules. Helpful for onboarding and keeps the domain model explicit. Just merged upstream/main to resolve the conflicts in the scraper files — the connector pattern takes precedence over the direct JSearch calls on main. Let me know if you have questions or want me to split anything up. |
|
I will need to understand whether this feature is required at this moment. The PR changes are too big so will take some time for me to review, also the target branch need to be dev |
3 participants
Hi,
Based on the projects main branch,
I'd like to contribute a set of new features, mentioned here as an high overview:
Allium domain specifications are also introduced that helps LLMs to understand the application scope and saves a lot of context windows.
For a detailed review please study the commit comments.
If there are any open questions, please let me know.
PS: Many thanks to @tburny for helping me out with the contribution