feat(model): add ARK SDK LLM and Embedding model support#46
feat(model): add ARK SDK LLM and Embedding model support#46sjy3 merged 3 commits intovolcengine:mainfrom
Conversation
- Add ARK SDK-based LLM (model/ark.go) using volcengine-go-sdk, supporting chat completion, streaming, tool calls, reasoning content, and thinking config - Add Embedder interface (model/embedding.go) and ARK embedding implementation (model/ark_embedding.go) for text vectorization via ARK SDK - Extract shared conversion utilities into model/convert.go (mapFinishReason, maybeAppendUserContent, extractTextFromContent, convertFunctionParameters) - Refactor model/openai.go to use shared utilities from convert.go - Add comprehensive unit tests (ark_test.go, ark_embedding_test.go) - Add EmbeddingModelConfig to configs and embedding defaults to common - Update agent/llmagent to support 'ark' model provider selection
There was a problem hiding this comment.
Pull request overview
This PR adds native Volcengine ARK SDK integration for LLM and Embedding models, replacing reliance on OpenAI-compatible HTTP endpoints. The implementation introduces ARK-specific model providers while maintaining backward compatibility with the existing OpenAI implementation through a provider selection mechanism.
Changes:
- Introduces ARK SDK-based LLM and Embedding model implementations with support for API Key and AK/SK authentication
- Refactors shared conversion utilities from
openai.gotomodel/convert.gofor reuse across providers - Adds provider-based routing in
agent/llmagentto select between "ark" and "openai" (default) implementations
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| model/embedding.go | Defines the Embedder interface for text embedding models |
| model/convert.go | Extracts shared conversion utilities (finish reason mapping, function parameters, content extraction) for reuse |
| model/ark.go | Implements ARK SDK LLM with chat completion, streaming, tool calls, reasoning content support |
| model/ark_embedding.go | Implements ARK SDK embedding model with batch embedding and configurable dimensions |
| model/ark_test.go | Unit tests for ARK LLM covering authentication, generation, streaming, tool calls, and reasoning |
| model/ark_embedding_test.go | Unit tests for ARK embedding covering single/batch embedding, dimension override, and error handling |
| model/openai.go | Refactored to use shared utilities from convert.go, removing duplicate code |
| agent/llmagent/agent.go | Adds ModelProvider config field with switch-based routing to ARK or OpenAI models |
| configs/model.go | Adds EmbeddingModelConfig struct with dimension configuration and env variable mapping |
| configs/configs.go | Initializes EmbeddingModelConfig in global config structure |
| common/defaults.go | Defines embedding model default values (name, API base, dimensions) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
please fix the :model/ark.go:554:34: invalid operation: usage.PromptTokensDetails != nil (mismatched types "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model".PromptTokensDetail and untyped nil) |
@sjy3 Fixed. |
|
LGTM |
Description
Add native Volcengine ARK SDK integration for LLM and Embedding models, replacing the dependency on OpenAI-compatible HTTP endpoints for ARK services.
Changes
New files:
model/ark.go— ARK SDK LLM implementation (chat completion, streaming, tool calls, reasoning content, thinking config)model/embedding.go—Embedderinterface definitionmodel/ark_embedding.go— ARK SDK embedding implementationmodel/convert.go— Shared conversion utilities extracted fromopenai.gomodel/ark_test.go— ARK LLM unit testsmodel/ark_embedding_test.go— ARK Embedding unit testsModified files:
model/openai.go— Refactored to use shared utilities fromconvert.goagent/llmagent/agent.go— Support"ark"model provider viaModelProviderconfigconfigs/model.go— AddEmbeddingModelConfigstructconfigs/configs.go— InitializeEmbeddingModelConfigin global configcommon/defaults.go— Add embedding model defaultsHighlights
volcengine-go-sdk/service/arkruntimedirectly, supporting AK/SK and API Key auth, thinking/reasoning config, streaming with tool call accumulationEmbedderinterface with batch embedding, configurable dimensionsagent/llmagentnow routes to ARK or OpenAI model based onModelProviderfield (default:"openai")mapFinishReason,maybeAppendUserContent,convertFunctionParameters, etc.) extracted tomodel/convert.gofor reuse