Skip to content

# feat(schedule): implement full GTFS import pipeline for Schedule module #6#7

Open
Gepacam wants to merge 4 commits intomainfrom
feature/schedule-import-#6
Open

# feat(schedule): implement full GTFS import pipeline for Schedule module #6#7
Gepacam wants to merge 4 commits intomainfrom
feature/schedule-import-#6

Conversation

@Gepacam
Copy link
Collaborator

@Gepacam Gepacam commented Nov 6, 2025

Description

  • Adds complete GTFS Schedule import functionality covering all 9 core files:
    agency.txt, routes.txt, stops.txt, trips.txt, calendar.txt,
    calendar_dates.txt, stop_times.txt, shapes.txt, and feed_info.txt.
  • Performs referential integrity validation and atomic insertion within a single transaction.
  • Implements error handling with line/file context for invalid references or missing fields.
  • Provides end-to-end test (tests/test_import_full_gtfs.py) that auto-generates synthetic GTFS feed and verifies successful ingestion.

Changes

  • Updated gtfs/utils/schedule.py — full import pipeline supporting required + optional GTFS files.
  • Added tests/test_import_full_gtfs.py — integration test creating 9 CSVs and ZIP in tmp_gtfs/.
  • Retained test_import_gtfs.py as a lightweight smoke test for minimal GTFS feeds.
  • Verified proper model mapping and transaction handling for all Schedule entities.

Subissue


Checklist

  • Supports both required and optional GTFS files (total 9 models).
  • Validates referential integrity and type consistency.
  • Reports errors with file/line context when applicable.
  • All tests pass locally

Note: A minimal version of this importer (supporting 4 models only) is still available under tests/test_import_gtfs.py for lightweight validation. It is fully optional and removable once CI integration is complete.

Implements core GTFS Schedule entities (agency, routes, trips, stops, stop_times, shapes, calendar, calendar_dates, feed_info)
with ForeignKeys and UniqueConstraints as specified in GTFS v2.0.

Migrations were created and applied successfully under PostGIS backend.

Closes #3.
…CPKs per academic guidance

Implements deterministic CRUD test coverage for the GTFS Schedule module, ensuring referential integrity across
AgencySchedule, CalendarSchedule, and ShapeSchedule.
Composite Primary Keys (CPKs) were intentionally not implemented, per academic guidance from course instructor,
to maintain compatibility with Django ORM and simplify admin integration.
…gration #5

Adds management command `create_schedule_fixtures` to generate a minimal, deterministic GTFS Schedule dataset.
Includes --seed and --output options, produces a tiny, loaddata-friendly JSON, and is exercised by a pytest test.

- Shapes PKs are numeric to align with Django's AutoField/BigAutoField.
- Tests run without PostGIS by using the non-GIS test settings.
- Validated end-to-end via `pytest -v tests/test_fixtures.py`.
#6

Implements complete GTFS Schedule import pipeline supporting all 9 core files:
agency, routes, stops, trips, calendar, calendar_dates, stop_times, shapes, and feed_info.

Validates referential integrity across related models and performs bulk inserts under a single transaction.
Includes integration test (test_import_full_gtfs.py) generating synthetic GTFS ZIP in tmp_gtfs/ to verify end-to-end ingestion.
@Gepacam Gepacam requested a review from fabianabarca November 6, 2025 17:36
@Gepacam Gepacam self-assigned this Nov 6, 2025
Gepacam added a commit that referenced this pull request Nov 7, 2025
…validation #7

Implements export_gtfs_schedule() to serialize all Schedule models into GTFS v2.0-compliant text files and ZIP package.
Includes deterministic validation with synthetic data and metadata embedding.
MBTA feed import/export remains experimental due to dataset size.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant