Skip to content

Conversation

@aaarendt
Copy link
Contributor

@aaarendt aaarendt commented Dec 19, 2025

Overview

This PR has scripts to deploy a Python handler to AWS Lambda that leverages native functionality of the existing api. The geospatial heavy operations require large libraries that were too cumbersome for Lambda, so many changes are implemented here to simplify and avoid use of geopandas. This includes relying on PostGIS functionality at the db backend, and relying on the client side to query geometries and convert back to geospatial objects, thereby avoiding needing geopandas in the Lambda handler itself.

Performance issues were discovered along the way since this is the first time doing operations on the 29 GB layers table. Specifically, all_instruments query should use EXISTS instead of DISTINCT with join which vastly improves performance.

Throughout I updated the handling of credentials following #204 .
This code was primarily developed using AI-assistance.

Supports deployment as an AWS Lambda function using Docker containers. This enables serverless execution with significantly reduced package size (~200MB vs several GB) while maintaining core database functionality.

This is a lightweight deployment that excludes GDAL/rasterio/geopandas for smaller containers, for now. Secure database credential management achieved through AWS Secrets Manager.
…dding error handling for geometric operations
…h for storing credentials as environment variables
@aaarendt aaarendt marked this pull request as draft December 19, 2025 18:02
@codecov
Copy link

codecov bot commented Dec 19, 2025

Codecov Report

❌ Patch coverage is 43.10954% with 322 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.68%. Comparing base (f8af2f8) to head (98b83ac).

Files with missing lines Patch % Lines
snowexsql/lambda_client.py 11.46% 193 Missing ⚠️
snowexsql/lambda_handler.py 64.22% 78 Missing ⚠️
snowexsql/api.py 60.76% 51 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #206       +/-   ##
===========================================
- Coverage   75.88%   59.68%   -16.20%     
===========================================
  Files          21       23        +2     
  Lines         651     1146      +495     
===========================================
+ Hits          494      684      +190     
- Misses        157      462      +305     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aaarendt aaarendt marked this pull request as ready for review December 22, 2025 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants