Skip to content

feat(server): (Do Not Merge) POC - implement hooks for server#110

Draft
lukeroantreeONS wants to merge 1 commit intomainfrom
POC_server_hooks_ratelimiter
Draft

feat(server): (Do Not Merge) POC - implement hooks for server#110
lukeroantreeONS wants to merge 1 commit intomainfrom
POC_server_hooks_ratelimiter

Conversation

@lukeroantreeONS
Copy link
Collaborator

Quick proof-of-concept for extending the VectorStore's idea of 'hooks' to the server.

✨ Summary

Do Not Merge - proof of concept only.

I had some time on a train journey, so put together this proposal for extending the idea of user-defined hooks to the server, and showed in a demo that this can allow them to add customisations like rate-limiting themselves.

As part of this, I split the server module's start_api() method into setup_api() (which returns a FastAPI application) and start_api() which runs it with Uvicorn.

These changes show the main idea behind the proposal, but in order to get the slowapi ratelimiter working I had to tweak the endpoint definitions slightly (making the request parameter explicit instead of implicit, removing the response Pydantic object type hinting as ratelimiter hijacks the response object). Some of these changes (removal of the type hinting / validation) can be avoided by configuring the ratelimiter more carefully - I didn't want to devote time to that while it's still just a proposal.

As this includes breaking, and un-discussed, this draft PR is only to show the idea to facilitate discussion, not to be merged.

📜 Changes Introduced

  • feat: user-specified hooks for the FastAPI server
  • chore: small refactor of the server module

✅ Checklist

Please confirm you've completed these checks before requesting a review.

  • Code passes linting with Ruff
  • Security checks pass using Bandit
  • API and Unit tests are written and pass using pytest
  • Terraform files (if applicable) follow best practices and have been validated (terraform fmt & terraform validate)
  • DocStrings follow Google-style and are added as per Pylint recommendations
  • Documentation has been updated if needed

🔍 How to Test

  1. build the package with uv build

  2. install slowapi ( uv pip install slowapi )

  3. copy the fake soc dataset in to the testbed/ directory

  4. run the example script ( cd testbed; uv run --script ./server_hooks_slowapi_ratelimiter_poc.py )

  5. open the swagger page, go to the search endpoint 'Try It Out', execute 3 times (checking the output after each). Expect 2 normal outputs and a rate limit error on the third (if 3 performed in under a minute).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant