feat(llm): add periodic availability checking for LLM clients #3022
+466
−16
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add background availability checking with configurable interval - Cache availability state to avoid blocking API calls on each request - Only initialize LLM client when rag.llm.type matches the client name - Add destroy() method for proper cleanup of timeout tasks - Add preDestroy configuration in fess_llm.xml - Ollama client now validates configured model exists - Add rag.llm.availability.check.interval configuration (default: 60s) - Update default model versions (gpt-5-mini, gemini-2.5-flash) Co-Authored-By: shinsuke <shinsuke@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds periodic background availability checking for all LLM clients (OpenAI, Gemini, Ollama) to improve performance by caching availability state and reducing blocking API calls. Each client now performs availability checks at a configurable 60-second interval (configurable via rag.llm.availability.check.interval) and only initializes when the configured rag.llm.type matches the client's name.
Changes:
- Added background availability checking with cached state using
TimeoutManagerfor periodic updates - Implemented selective client initialization based on configured LLM type to reduce resource usage
- Enhanced Ollama availability check to verify the configured model exists in the server
- Updated default model versions to
gpt-5-mini(OpenAI) andgemini-2.5-flash(Gemini)
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/main/java/org/codelibs/fess/llm/openai/OpenAiLlmClient.java | Added periodic availability checking, caching, and resource cleanup for OpenAI client |
| src/main/java/org/codelibs/fess/llm/gemini/GeminiLlmClient.java | Added periodic availability checking, caching, and resource cleanup for Gemini client |
| src/main/java/org/codelibs/fess/llm/ollama/OllamaLlmClient.java | Added periodic availability checking with model validation, caching, and resource cleanup for Ollama client |
| src/main/resources/fess_llm.xml | Added preDestroy lifecycle hooks for all three LLM clients |
| src/main/resources/fess_config.properties | Added availability check interval configuration and updated default model versions |
| src/main/java/org/codelibs/fess/mylasta/direction/FessConfig.java | Added configuration getters for availability check interval and updated model version documentation |
| src/main/java/org/codelibs/fess/mylasta/action/FessLabels.java | Added chat phase labels for UI status messages |
| src/test/java/org/codelibs/fess/llm/openai/OpenAiLlmClientTest.java | Added getLlmType() override to test client for selective initialization support |
| src/test/java/org/codelibs/fess/llm/gemini/GeminiLlmClientTest.java | Added getLlmType() override to test client for selective initialization support |
| src/test/java/org/codelibs/fess/llm/ollama/OllamaLlmClientTest.java | Added getLlmType() override to test client for selective initialization support |
Comments suppressed due to low confidence (2)
src/main/java/org/codelibs/fess/mylasta/direction/FessConfig.java:1
- The model version
gpt-5-minidoes not exist. As of January 2025, OpenAI's GPT-5 has not been released. Valid models includegpt-4o,gpt-4-turbo, orgpt-3.5-turbo. Update to a valid model version.
/*
src/main/java/org/codelibs/fess/mylasta/direction/FessConfig.java:1
- The model version
gemini-2.5-flashdoes not exist. As of January 2025, valid Gemini models includegemini-1.5-pro,gemini-1.5-flash, andgemini-pro. Update to a valid model version.
/*
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds periodic availability checking for all LLM clients (OpenAI, Gemini, Ollama) to improve performance and provide proactive status monitoring. The availability state is now cached and updated in the background, eliminating blocking API calls on each availability check request.
Changes Made
Core Implementation
TimeoutManagercachedAvailabilityfield with volatile semantics for thread-safe reads without synchronization overheadrag.llm.typematches the client name, reducing unnecessary resource usageLLM Client Updates
Configuration
rag.llm.availability.check.intervalproperty (default: 60 seconds)gpt-5-minigemini-2.5-flashResource Management
destroy()method to all LLM clients for proper cleanuppreDestroyconfiguration infess_llm.xmlfor lifecycle managementTesting
Breaking Changes
None. The availability checking is backward-compatible:
Additional Notes
rag.chat.enabled=true)