Feature Request: Progress Bars

[olirice]

The lack of feedback during Collection.upsert and Collection.create_index is a bad DX.

It would be great to get some progress bars but I haven't been able to get them working properly in notebooks and shell environments.

If anyone has experience adding them this would be a great community contribution

upsert

vecs/src/vecs/collection.py

Line 108 in 87ed2d3

for chunk in flu(vectors).chunk(chunk_size):

but it can't always assume that vectors is sized. Maybe a runtime check to see if *vectors* has a known length and providing it to the progress bar if known would be best

create_index

vecs/src/vecs/collection.py

Lines 346 to 350 in 87ed2d3

	stmt = postgresql.insert(clone_table).from_select(
	self.table.c, select(self.table)
	)
	stmt = stmt.on_conflict_do_nothing()
	sess.execute(stmt)

this one is a little more involved as it'll have to introduce client side keyset pagination on the id primary key to get feedback to python

[sahusiddharth]

@olirice I would like to work on it.

can please point me to contributing guideline for this repo I was find it

[olirice]

great!

we don't have official contributing guidelines

Broadly you'll need

• python 3.7+ (and be compatible with 3.7)
• docker

setup a virtualenv

python -m venv venv
source venv/bin/activate
pip install -e ".[dev,docs,text_embedding]"

install the pre-commit hooks

pre-commit install

at that point you should be able to run

pytest

and have it pass. Note: on the first run a docker container will be spinning up so it could take a couple minutes. After that It'll be much faster.

We have 100% code coverage as a requirement so any new lines/functions must be covered. You can check if what you introduce is covered using

pytest --cov=vecs --cov-report=html -vv

to generate a report, followed by

open htmlcov/index.html 

to see the coverage report

all new functions/methods should conform to the current documentation style

let me know if you have any trouble