Skip to content

fix: Pyarrow codebase dependencies not removed#123

Open
frayle-ons wants to merge 3 commits intomainfrom
122-bug-pyarrow-code-dependencies-in-codebase
Open

fix: Pyarrow codebase dependencies not removed#123
frayle-ons wants to merge 3 commits intomainfrom
122-bug-pyarrow-code-dependencies-in-codebase

Conversation

@frayle-ons
Copy link
Contributor

📌 Pull Request Template

✨ Summary

As part #117 we removed several non-immediate dependencies from the package including PyArrow, PyArrow also corresponded to several stakeholder requests that this library was causing dependency issues with users' existing codebases.

However, several of our Polar's datamframe operations still rely on the use of the PyArrow back end leading to ModuleNotFoundError errors in both the VectorStore search() and reverse_search() methods.

These have been resolved by using alternate Polars methods that do not require PyArrow under the hood.

Additionally, I've updated the package version number in the uv lock file and also set name mappings in for deptry for google-genai and ollama optional installations.

The code changes now allow us to run all the DEMO folder Jupyter notebooks without error, which seems like a good test.

📜 Changes Introduced

  • Feature implementation (feat:) / bug fix (fix:) / refactoring (chore:) / documentation (docs:) / testing (test:)
  • fix: removed polars code that depended on Pyarrow in search and reverse_search method, primarily concerning converting pandas data frames to polars data frames
  • Updated uv lock file
  • Updated pyproject.toml to successfully map optional dependency names for google and ollama to non trivial foo-bar import names

✅ Checklist

  • Code passes linting with Ruff
  • Security checks pass using Bandit
  • API and Unit tests are written and pass using pytest
  • Terraform files (if applicable) follow best practices and have been validated (terraform fmt & terraform validate)
  • DocStrings follow Google-style and are added as per Pylint recommendations
  • Documentation has been updated if needed

🔍 How to Test

Running each of the notebooks after building the branch from source showed no more issues. The changes introduced to the pyproject.toml also smoothed out issues seem before with pre-commit deputy

@frayle-ons frayle-ons linked an issue Feb 5, 2026 that may be closed by this pull request
@frayle-ons frayle-ons marked this pull request as ready for review February 5, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Pyarrow code dependencies in codebase

1 participant