Skip to content

feat(examples): add custom HTTP embedding example for LM Studio / Ollama#149

Open
cluster2600 wants to merge 5 commits intoalibaba:mainfrom
cluster2600:feat/lmstudio-custom-http-embedding
Open

feat(examples): add custom HTTP embedding example for LM Studio / Ollama#149
cluster2600 wants to merge 5 commits intoalibaba:mainfrom
cluster2600:feat/lmstudio-custom-http-embedding

Conversation

@cluster2600
Copy link
Contributor

Summary

This PR adds a self-contained example showing how to use any OpenAI-compatible HTTP embedding endpoint (LM Studio, Ollama, vLLM, LocalAI, …) as the embedding source in zvec.

What's added

examples/custom_http_embedding.py

HTTPEmbeddingFunction A zero-dependency class (stdlib only) that calls any /v1/embeddings endpoint, caches results with @lru_cache, and satisfies the DenseEmbeddingFunction protocol.
Collection setup HNSW index with cosine similarity, dimension auto-detected from the server response.
Insert + query 5 sample documents embedded on the fly, then a semantic search query.
CLI interface --base-url, --model, --api-key, --collection-path flags for easy customisation.
README-style header Step-by-step instructions for LM Studio and Ollama embedded at the top of the file.

Usage

# LM Studio (default)
python examples/custom_http_embedding.py

# Ollama
python examples/custom_http_embedding.py \
    --base-url http://localhost:11434 \
    --model nomic-embed-text

Motivation

The existing extensions (OpenAIDenseEmbedding, etc.) require the openai package and are primarily designed for cloud APIs. Many developers want to use local inference servers without extra dependencies. This example shows the pattern using only Python stdlib, making it easy to adapt or inline.

Testing

The example runs end-to-end against a live LM Studio instance on localhost:1234. No new test infrastructure is required for a standalone script.

@CLAassistant
Copy link

CLAassistant commented Feb 19, 2026

CLA assistant check
All committers have signed the CLA.

@Cuiyus
Copy link
Collaborator

Cuiyus commented Feb 26, 2026

Thank you for your submission!

This service-oriented model for invocation, which helps zvec achieve the RAG capability, is what we currently lack.
You can implement your HttpEmbedingFunction(OllmaEmbedingFunction) by inheriting DenseEmbedingFunction in the directory python/zvec/extension. This will make it easier for more users to use!

https://zvec.org/api-reference/python/extension/#zvec.extension.DenseEmbeddingFunction

@cluster2600
Copy link
Contributor Author

Thanks for the feedback! Moved the implementation into python/zvec/extension/http_embedding_function.py as HTTPDenseEmbedding, inheriting from DenseEmbeddingFunction. It's now exported from zvec.extension and the example imports it from there instead of defining the class inline.

Move the HTTP embedding implementation from the example script into
python/zvec/extension/ as HTTPDenseEmbedding, inheriting from
DenseEmbeddingFunction. The example now imports from zvec.extension
instead of defining the class inline.

Signed-off-by: Maxime <maxime@cluster2600.com>
Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com>
@cluster2600 cluster2600 force-pushed the feat/lmstudio-custom-http-embedding branch from 31f67ed to 9a81b28 Compare February 26, 2026 09:11
Move zvec imports to top-level, add noqa for print statements,
replace os.path.exists with pathlib, fix import sorting.

Signed-off-by: Maxime <maxime@cluster2600.com>
Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com>
Signed-off-by: Maxime <maxime@cluster2600.com>
Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com>
The vector_column_indexer_test failure is a known flaky assertion in
hnsw_streamer_entity.h, unrelated to Python-only changes in this PR.

Signed-off-by: Maxime <maxime@cluster2600.com>
Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants