Skip to content

Commit 3334bac

Browse files
crpietschmanncrpietschmann
authored
Merge pull request #73 from Build5Nines/dev
v2.1.1
2 parents bd69ace + d5647ff commit 3334bac

File tree

22 files changed

+385
-72
lines changed

22 files changed

+385
-72
lines changed

‎.github/workflows/ghpages-mkdocs.yml‎

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
name: Deploy MKDocs site to Pages
1+
name: Build MKDocs Site
22

33
on:
4-
push:
4+
pull_request:
55
branches:
66
- main
7+
- dev
78
paths:
89
- .github/workflows/ghpages-mkdocs.yml
910
- docs/**
@@ -63,14 +64,3 @@ jobs:
6364
uses: actions/upload-pages-artifact@v3
6465
with:
6566
path: ./docs/_site
66-
67-
deploy:
68-
runs-on: ubuntu-latest
69-
needs: build
70-
environment:
71-
name: github-pages
72-
url: ${{ steps.deployment.outputs.page_url }}
73-
steps:
74-
- name: Deploy to GitHub Pages
75-
id: deployment
76-
uses: actions/deploy-pages@v4

‎.github/workflows/mkdocs-build.yml‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
name: Deploy MKDocs Site to Pages
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
paths:
8+
- .github/workflows/ghpages-mkdocs.yml
9+
- docs/**
10+
- mkdocs.yml
11+
workflow_dispatch:
12+
13+
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
14+
permissions:
15+
contents: read
16+
pages: write
17+
id-token: write
18+
19+
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
20+
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
21+
concurrency:
22+
group: "pages"
23+
cancel-in-progress: false
24+
25+
jobs:
26+
build:
27+
runs-on: ubuntu-latest
28+
29+
steps:
30+
- name: Checkout code
31+
uses: actions/checkout@v3
32+
with:
33+
fetch-depth: 0 # Fetch all history for all branches and tags, not just the default branch.
34+
# This is needed to ensure that the commit SHA is available for the deployment.
35+
# See
36+
sparse-checkout: |
37+
docs
38+
mkdocs.yml
39+
.github/workflows/ghpages-mkdocs.yml
40+
41+
- name: Setup pages
42+
id: pages
43+
uses: actions/configure-pages@v5
44+
45+
46+
- name: Set up Python
47+
uses: actions/setup-python@v4
48+
with:
49+
python-version: '3.x' # Use the latest version of Python 3
50+
51+
- name: Install dependencies
52+
run: |
53+
python -m pip install --upgrade pip
54+
pip install -r ./docs/requirements.txt
55+
56+
- name: Build documentation
57+
# Outputs to the './_site' directory by default
58+
run: |
59+
mkdocs build --site-dir ./_site --config-file ./docs/mkdocs.yml
60+
61+
- name: Upload artifact
62+
# Automatically upload an artifact from the './_site' directory by default
63+
uses: actions/upload-pages-artifact@v3
64+
with:
65+
path: ./docs/_site
66+
67+
deploy:
68+
runs-on: ubuntu-latest
69+
needs: build
70+
environment:
71+
name: github-pages
72+
url: ${{ steps.deployment.outputs.page_url }}
73+
steps:
74+
- name: Deploy to GitHub Pages
75+
id: deployment
76+
uses: actions/deploy-pages@v4

‎CHANGELOG.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## v2.1.1
9+
10+
Add:
11+
12+
- Add optional `filter` parameter to `.Search` and `.SearchAsync` methods that is of type `Func<TMetadata, bool>` that is called for each text item in the database for more advanced filtering prior to performing vector similarity search and returning results. If undefined or `null` it's ignored.
13+
814
## v2.1.0
915

1016
Add:

‎docs/docs/concepts/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Concepts
33
description: Understand the core concepts behind SharpVector, from vector similarity to embedding strategies and in-memory architecture.
44
---
5-
# Concepts
5+
# :octicons-light-bulb-24: Concepts
66

77
## What is a Vector Database?
88

‎docs/docs/embeddings/index.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
# Embeddings
1+
---
2+
title: Embeddings
3+
---
4+
# :fontawesome-solid-square-binary: Embeddings
25

36
`Build5Nines.SharpVector` includes the following support for using AI Models to generate the text embeddings for the vector database instead of generating them locally. The use of an AI Embeddings model can greatly increase the quality of the semantic search.
47

‎docs/docs/get-started/data-management/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Data Management
33

44
---
5-
# Data Management
5+
# :material-database-edit-outline: Data Management
66

77
Since `Build5Nines.SharpVector` is a database, it also has data management methods available. These methods enable you to add, remove, and update the text documents that are vectorized and indexed within the semantic database.
88

‎docs/docs/get-started/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Get Started
33
description: Get up and running with SharpVector in minutes. Learn how to install, initialize, and begin storing and searching vectorized text data.
44
---
5-
# Get Started
5+
# :octicons-rocket-24: Get Started
66

77
It's really easy to get started with using `Build5Nines.SharpVector`. Simply follow the below steps.
88

‎docs/docs/get-started/metadata/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Metadata
33
---
4-
# Metadata
4+
# :material-database-cog-outline: Metadata
55

66
The `Build5Nines.SharpVector` vector database enables semantic search for `Text` that is stored in the database. Being able to semantically search text is an extremely useful way to lookup more information related to the text. For this purpose, `Metadata` is stored alongside the `Text` within the vector database. This way, when `Text` is found when performing a semantic search, then the matching `Metadata` is also retrieved.
77

‎docs/docs/get-started/search/index.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
---
2+
title: Semantic Search
3+
---
4+
# :material-file-search: Semantic Search
5+
6+
Once text items and their associated metadata have been added to the vector database, the database can be used for semantic search to find matching text items for a given query.
7+
8+
The `BasicMemoryVectorDatabase` and `MemoryVectorDatabase<>` classes both contain `.Search` and `.SearchAsync` methods that can be used to perform semantic search on the database:
9+
10+
=== "Sync"
11+
12+
```csharp
13+
var query = "some text to search";
14+
var results = vdb.Search(query);
15+
```
16+
17+
=== "Async"
18+
19+
```csharp
20+
var query = "some text to search";
21+
var results = await vdb.SearchAsync(query);
22+
```
23+
24+
## Metadata Filters
25+
26+
The `.Search` and `.SearchAsync` methods also include the ability to pre-filter the search results based on a boolean evaluation of the `Metadata` for the text item. This check is run before the vector similarity search is performed, and can help increase search performance on large datasets.
27+
28+
Here are a couple examples of using the `filter` parameter to perform `Metadata` filtering when performing semantic searches:
29+
30+
=== "Sync"
31+
32+
```csharp
33+
var vdb = new BasicMemoryVectorDatabase();
34+
35+
// load text and metadata into database
36+
37+
var query = "some text to search";
38+
var results = vdb.Search(
39+
query,
40+
filter: (metadata) => {
41+
// perform some operation to check metadata
42+
// return true or false
43+
return metadata.Contains("B59");
44+
}
45+
);
46+
```
47+
48+
=== "Async"
49+
50+
```csharp
51+
var vdb = new MemoryVectorDatabase<Person>();
52+
53+
// load text and metadata into database
54+
55+
var query = "some text to search";
56+
var results = vdb.SearchAsync(
57+
query,
58+
filter: async (metadata) => {
59+
// perform some operation to check metadata
60+
// return true or false
61+
return metadata.LastName == "Pietschmann";
62+
}
63+
);
64+
```
65+
66+
!!! info "OpenAI and Ollama Support"
67+
68+
This functionality works the same with both [:simple-openai: OpenAI and :simple-ollama: Ollama supported vector databases](../../embeddings/index.md) too.
69+
70+
## Paging
71+
72+
The `.Search` and `.SearchAsync` methods also include the ability to perform paging on the text items returned from the semantic search. This is performed after the similarity search and the `filter` has been applied to the search results. This is done using the optional `pageCount` and `pageIndex` paramters.
73+
74+
Here are a couple examples of using the `pageCount` and `pageIndex` parameters to perform paging with the semantic search results:
75+
76+
=== "Sync"
77+
78+
```csharp
79+
var vdb = new BasicMemoryVectorDatabase();
80+
81+
// load text and metadata into database
82+
83+
var query = "some text to search";
84+
var results = vdb.Search(
85+
query,
86+
pageIndex: 0, // return first page of results (default: 0)
87+
pageCount: 6 // limit length of this page of results (default: unlimited)
88+
);
89+
```
90+
91+
=== "Async"
92+
93+
```csharp
94+
var vdb = new MemoryVectorDatabase<Person>();
95+
96+
// load text and metadata into database
97+
98+
var query = "some text to search";
99+
var results = vdb.SearchAsync(
100+
query,
101+
pageIndex: 0, // return first page of results (default: 0)
102+
pageCount: 6 // limit length of this page of results (default: unlimited)
103+
);
104+
```
105+
106+
The `pageIndex` and `pageIndex` paramters are optional, and can be used individually or together.

‎docs/docs/index.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Discover
33
description: The lightweight, in-memory, semantic search, text vector database for .NET that powers intelligent search and recommendation features.
44
---
5-
# Discover Build5Nines.SharpVector
5+
# :fontawesome-regular-compass: Discover Build5Nines.SharpVector
66

77
**Build5Nines.SharpVector** is the lightweight, in-memory, semantic search, text vector database built for .NET applications. It enables fast and flexible vector-based similarity search for text data — ideal for search engines, recommendation systems, semantic analysis, and AI-enhanced features.
88

@@ -27,6 +27,8 @@ Vector databases are used with Semantic Search and [Generative AI](https://build
2727

2828
While there are lots of large databases that can be used to build Vector Databases (like Azure CosmosDB, PostgreSQL w/ pgvector, Azure AI Search, Elasticsearch, and more), there are not many options for a lightweight vector database that can be embedded into any .NET application. Build5Nines SharpVector is the lightweight in-memory Text Vector Database for use in any .NET application that you're looking for!
2929

30+
<!-- [:material-file-pdf-box: Download documentation](/build5nines-sharpvecto.pdf){ .md-button .md-button--secondary } -->
31+
3032
---
3133

3234
> "For the in-memory vector database, we're using Build5Nines.SharpVector, an excellent open-source project by Chris Pietschmann. SharpVector makes it easy to store and retrieve vectorized data, making it an ideal choice for our sample RAG implementation."

0 commit comments

Comments
 (0)