Skip to content

feat(model): add ARK SDK LLM and Embedding model support#46

Merged
sjy3 merged 3 commits intovolcengine:mainfrom
wucm667:feat/ark-model-and-embedding
Feb 27, 2026
Merged

feat(model): add ARK SDK LLM and Embedding model support#46
sjy3 merged 3 commits intovolcengine:mainfrom
wucm667:feat/ark-model-and-embedding

Conversation

@wucm667
Copy link
Contributor

@wucm667 wucm667 commented Feb 13, 2026

Description

Add native Volcengine ARK SDK integration for LLM and Embedding models, replacing the dependency on OpenAI-compatible HTTP endpoints for ARK services.

Changes

New files:

  • model/ark.go — ARK SDK LLM implementation (chat completion, streaming, tool calls, reasoning content, thinking config)
  • model/embedding.goEmbedder interface definition
  • model/ark_embedding.go — ARK SDK embedding implementation
  • model/convert.go — Shared conversion utilities extracted from openai.go
  • model/ark_test.go — ARK LLM unit tests
  • model/ark_embedding_test.go — ARK Embedding unit tests

Modified files:

  • model/openai.go — Refactored to use shared utilities from convert.go
  • agent/llmagent/agent.go — Support "ark" model provider via ModelProvider config
  • configs/model.go — Add EmbeddingModelConfig struct
  • configs/configs.go — Initialize EmbeddingModelConfig in global config
  • common/defaults.go — Add embedding model defaults

Highlights

  • ARK LLM: Uses volcengine-go-sdk/service/arkruntime directly, supporting AK/SK and API Key auth, thinking/reasoning config, streaming with tool call accumulation
  • ARK Embedding: Implements Embedder interface with batch embedding, configurable dimensions
  • Provider selection: agent/llmagent now routes to ARK or OpenAI model based on ModelProvider field (default: "openai")
  • Refactor: Common logic (mapFinishReason, maybeAppendUserContent, convertFunctionParameters, etc.) extracted to model/convert.go for reuse

- Add ARK SDK-based LLM (model/ark.go) using volcengine-go-sdk, supporting
  chat completion, streaming, tool calls, reasoning content, and thinking config
- Add Embedder interface (model/embedding.go) and ARK embedding implementation
  (model/ark_embedding.go) for text vectorization via ARK SDK
- Extract shared conversion utilities into model/convert.go (mapFinishReason,
  maybeAppendUserContent, extractTextFromContent, convertFunctionParameters)
- Refactor model/openai.go to use shared utilities from convert.go
- Add comprehensive unit tests (ark_test.go, ark_embedding_test.go)
- Add EmbeddingModelConfig to configs and embedding defaults to common
- Update agent/llmagent to support 'ark' model provider selection
Copilot AI review requested due to automatic review settings February 13, 2026 07:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds native Volcengine ARK SDK integration for LLM and Embedding models, replacing reliance on OpenAI-compatible HTTP endpoints. The implementation introduces ARK-specific model providers while maintaining backward compatibility with the existing OpenAI implementation through a provider selection mechanism.

Changes:

  • Introduces ARK SDK-based LLM and Embedding model implementations with support for API Key and AK/SK authentication
  • Refactors shared conversion utilities from openai.go to model/convert.go for reuse across providers
  • Adds provider-based routing in agent/llmagent to select between "ark" and "openai" (default) implementations

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
model/embedding.go Defines the Embedder interface for text embedding models
model/convert.go Extracts shared conversion utilities (finish reason mapping, function parameters, content extraction) for reuse
model/ark.go Implements ARK SDK LLM with chat completion, streaming, tool calls, reasoning content support
model/ark_embedding.go Implements ARK SDK embedding model with batch embedding and configurable dimensions
model/ark_test.go Unit tests for ARK LLM covering authentication, generation, streaming, tool calls, and reasoning
model/ark_embedding_test.go Unit tests for ARK embedding covering single/batch embedding, dimension override, and error handling
model/openai.go Refactored to use shared utilities from convert.go, removing duplicate code
agent/llmagent/agent.go Adds ModelProvider config field with switch-based routing to ARK or OpenAI models
configs/model.go Adds EmbeddingModelConfig struct with dimension configuration and env variable mapping
configs/configs.go Initializes EmbeddingModelConfig in global config structure
common/defaults.go Defines embedding model default values (name, API base, dimensions)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@sjy3
Copy link
Collaborator

sjy3 commented Feb 24, 2026

please fix the :model/ark.go:554:34: invalid operation: usage.PromptTokensDetails != nil (mismatched types "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model".PromptTokensDetail and untyped nil)

@wucm667
Copy link
Contributor Author

wucm667 commented Feb 25, 2026

please fix the :model/ark.go:554:34: invalid operation: usage.PromptTokensDetails != nil (mismatched types "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model".PromptTokensDetail and untyped nil)

@sjy3 Fixed. PromptTokensDetails in the ARK SDK is a value type struct (PromptTokensDetail), not a pointer, so it cannot be compared with nil. Changed to check CachedTokens > 0 directly. Local CI verification passed (build, lint, tests all green).

@sjy3
Copy link
Collaborator

sjy3 commented Feb 27, 2026

LGTM

@sjy3 sjy3 merged commit b368dc7 into volcengine:main Feb 27, 2026
2 checks passed
@wucm667 wucm667 deleted the feat/ark-model-and-embedding branch February 27, 2026 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants