都営バスデータ対応

[TinyKitten]

odptから恒久的にバス停位置とか系統の情報が取れる

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

✅ Successfully generated implementation plan.

💡 Use @coderabbitai <feedback> to replan

Examples

• Example 1
• Example 2

---

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!

[coderabbitai]

Coding Plan

Approach

Ticket Summary

Add support for fetching and storing Toei (Tokyo Metropolitan) bus data from the ODPT (Open Data for Public Transportation) API, including bus stop locations and route information, alongside the existing train/railway data.

Observations

Codebase Summary

The project is a Rust-based gRPC API service using Domain-Driven Design architecture with separate domain, infrastructure, presentation, and use-case layers. The system currently handles exclusively Japanese railway data (stations, lines, companies, train types) stored in PostgreSQL and loaded from CSV files during startup. The database schema uses 8 core tables with railway-specific terminology throughout. The gRPC service definitions are managed in an external Git submodule (TrainLCD/gRPCProto), and the API follows a repository pattern with clean separation of concerns.

Assumptions

Assumption 1: Transportation mode differentiation approach

Options Considered:

• Create parallel bus-specific tables (bus_companies, bus_routes, bus_stops) completely separate from train tables
• Add a transit_mode field to existing tables (lines, stations, companies) to differentiate trains from buses
• Hybrid approach: Add mode field and create mode-specific extension tables for non-shared attributes

Chosen Option: Add transit_mode field to existing tables

Rationale: This approach reuses existing infrastructure and allows both modes to coexist in the same domain model while keeping mode-specific logic contained. Since bus stops and train stations share most attributes (location, name, multilingual support), and bus routes and train lines have similar structures, extending the existing schema is more natural than duplicating the entire architecture. The ticket's phrasing ("恒久的に" - permanently) suggests this is a strategic addition rather than a temporary parallel system.

Assumption 2: ODPT data format handling

Options Considered:

• Store fetched ODPT JSON directly in the database with a JSONB column
• Transform ODPT JSON to CSV format to match existing import pipeline
• Parse ODPT data and insert directly into existing table structures

Chosen Option: Parse ODPT data and insert directly into existing tables

Rationale: The current CSV import is a startup mechanism, not the core data model. Fetching from ODPT should populate the database directly without intermediate CSV serialization, which would be unnecessary complexity. This aligns with adding a proper external data source integration.

Assumption 3: Data synchronization strategy

Options Considered:

• One-time import on startup like CSV files
• Periodic scheduled job that refreshes data
• On-demand refresh via admin endpoint

Chosen Option: Periodic scheduled job with configurable interval

Rationale: ODPT provides static bus route/stop data that changes infrequently but requires periodic updates. A background task that fetches and updates bus data (e.g., daily) ensures data stays current without manual intervention, following ODPT's guideline to update within one week of data changes.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Add ODPT API Client Infrastructure

Establish the foundation for fetching bus data from ODPT by adding HTTP client capabilities and the ODPT integration module.

Task 1: Add HTTP Client Dependency

Add the necessary HTTP client library to enable external API calls.

• Add reqwest (with async support and JSON feature) and tokio crate dependencies to stationapi/Cargo.toml
• Add serde_json for JSON parsing if not already present
• Ensure tokio runtime is configured with appropriate features for async HTTP

Task 2: Create ODPT Client Module

Create a new infrastructure module responsible for ODPT API communication.

• Create stationapi/src/infrastructure/odpt_client.rs module to encapsulate ODPT API interactions
• Implement authentication using ODPT API access token (read from environment variable or config)
• Create methods for fetching bus operators, bus routes (odpt:BusroutePattern), and bus stops (odpt:BusStopPole)
• Implement proper error handling for network failures, authentication errors, and API rate limits
• Add retry logic with exponential backoff for transient failures
• Parse JSON responses into intermediate Rust structures that mirror ODPT's data format

Task 3: Define ODPT Data Models

Create structures to represent ODPT API responses before transformation to domain entities.

• Create stationapi/src/infrastructure/odpt_models.rs with serde-compatible structs for ODPT JSON formats
• Define structures for BusroutePattern (route information, operator reference, pattern details)
• Define structures for BusStopPole (stop location, name, pole number, references to operators)
• Define structures for Operator (operator information, similar to railway companies)
• Include multilingual name fields (ja, en, etc.) matching ODPT's structure

🤖 Prompt for AI agents

Build the ODPT API integration infrastructure to fetch bus data.

Add reqwest, tokio, and serde_json dependencies to Cargo.toml

Create odpt_client.rs module with:
- Authentication using API token from environment
- Methods to fetch bus operators, routes (BusroutePattern), and stops
(BusStopPole)
- Error handling for network/auth failures and rate limits
- Retry logic with exponential backoff

Create odpt_models.rs with serde structs for:
- BusroutePattern (route info, operator reference, patterns)
- BusStopPole (location, name, pole number, operator references)
- Operator (company information)
- Include multilingual name fields (ja, en)

Phase 2: Extend Database Schema for Multi-Modal Support

Modify the database schema to accommodate both train and bus data within the existing table structures.

Task 1: Add Transit Mode Column to Core Tables

Extend existing tables to support transportation mode differentiation.

• Create database migration script in data/ directory (e.g., migrate_add_transit_mode.sql)
• Add transit_mode VARCHAR or ENUM column to companies table (values: 'railway', 'bus')
• Add transit_mode VARCHAR or ENUM column to lines table to distinguish bus routes from train lines
• Add transit_mode VARCHAR or ENUM column to stations table to distinguish bus stops from train stations
• Set default value to 'railway' for all existing records to maintain backward compatibility
• Add database indexes on transit_mode column for efficient filtering queries

Task 2: Add Bus-Specific Fields

Extend tables with nullable columns for bus-specific attributes that don't apply to trains.

• Add route_pattern TEXT column to lines table for bus route pattern identifiers (nullable, used only for buses)
• Add pole_number VARCHAR column to stations table for bus stop pole numbers (nullable, bus-specific)
• Consider adding operator_type column to companies table if bus operators have different categorization than railway companies
• Ensure all bus-specific columns are nullable so train records remain unaffected

Task 3: Update Data Import Logic

Modify the CSV import function to set transit_mode correctly for existing railway data.

• Update stationapi/src/main.rs CSV import function to explicitly set transit_mode='railway' when inserting data
• Ensure existing CSV import continues to work without modification to CSV files themselves
• Add SQL statements to the import that update the transit_mode field appropriately

🤖 Prompt for AI agents

Extend database schema to support both train and bus data.

Create migration script that:
- Adds transit_mode column (VARCHAR/ENUM: 'railway', 'bus') to companies, lines,
and stations tables
- Sets default value 'railway' for existing records
- Adds indexes on transit_mode columns
- Adds nullable bus-specific columns: route_pattern to lines, pole_number to
stations

Update CSV import in main.rs to:
- Explicitly set transit_mode='railway' when inserting railway data
- Maintain backward compatibility with existing CSV files

Phase 3: Extend Domain Model and Repositories

Update domain entities and data access layers to handle both transportation modes.

Task 1: Update Domain Entities

Modify entity definitions to include transit mode information.

• Update stationapi/src/domain/entity/station.rs to add transit_mode field to Station struct
• Update stationapi/src/domain/entity/line.rs to add transit_mode and optional route_pattern fields to Line struct
• Update stationapi/src/domain/entity/company.rs to add transit_mode field
• Add helper methods to entities for mode checking (e.g., is_bus(), is_railway())
• Ensure all existing entity constructors and methods handle the new fields appropriately

Task 2: Update Repository Implementations

Modify database query logic to include transit mode in SELECT, INSERT, and UPDATE operations.

• Update stationapi/src/infrastructure/station_repository.rs to include transit_mode in all SQL queries
• Update stationapi/src/infrastructure/line_repository.rs to include transit_mode and bus-specific fields
• Update stationapi/src/infrastructure/company_repository.rs to include transit_mode
• Add repository methods specifically for fetching bus data (e.g., get_bus_stops_by_route, get_bus_routes_by_operator)
• Ensure existing repository methods default to filtering by transit_mode='railway' if not specified, maintaining backward compatibility

Task 3: Create ODPT Data Transformer

Implement logic to transform ODPT response data into domain entities.

• Create stationapi/src/infrastructure/odpt_transformer.rs module
• Implement functions to convert ODPT BusStopPole to Station entities with transit_mode='bus'
• Implement functions to convert ODPT BusroutePattern to Line entities with transit_mode='bus'
• Implement functions to convert ODPT Operator to Company entities with transit_mode='bus'
• Handle coordinate transformation if ODPT uses different coordinate systems
• Map multilingual fields appropriately (ODPT's ja/en to the system's name fields)

🤖 Prompt for AI agents

Update domain model and repositories for multi-modal support.

Add transit_mode field to Station, Line, and Company entities
Add optional route_pattern to Line, add helper methods (is_bus, is_railway)

Update all repository implementations:
- Include transit_mode in all SQL queries (SELECT, INSERT, UPDATE)
- Add bus-specific methods: get_bus_stops_by_route, get_bus_routes_by_operator
- Default to filtering transit_mode='railway' for backward compatibility

Create odpt_transformer.rs to convert ODPT models to domain entities:
- BusStopPole → Station (transit_mode='bus')
- BusroutePattern → Line (transit_mode='bus')
- Operator → Company (transit_mode='bus')
- Handle coordinate transformation and multilingual field mapping

Phase 4: Extend gRPC API for Bus Data Access

Update the protobuf definitions and gRPC service implementation to expose bus data through the API.

Task 1: Update Protobuf Definitions

Modify the external protobuf repository to support bus queries.

• Initialize the proto Git submodule: git submodule update --init
• In the proto submodule, add transit_mode field to Station, Line, and Company message types in proto/stationapi.proto
• Add bus-specific optional fields (route_pattern to Line, pole_number to Station)
• Add new RPC methods or extend existing ones to support filtering by transit mode (e.g., get_bus_stops_by_route, get_bus_routes_by_operator)
• Consider adding GetBusRoutes and GetBusStopsByRoute RPC methods if bus-specific endpoints are needed
• Commit and push changes to the proto submodule repository

Task 2: Update gRPC Service Implementation

Modify the Rust gRPC controller to handle bus-related requests.

• Update stationapi/src/presentation/controller/grpc.rs to implement new or extended bus-related RPC methods
• Add logic to filter results by transit_mode parameter if provided in requests
• Ensure existing methods continue to return only railway data by default for backward compatibility
• Map domain entities (with transit_mode) to protobuf messages properly
• Handle optional bus-specific fields when serializing responses

Task 3: Update Build Configuration

Ensure protobuf compilation includes new fields and serde support.

• Update stationapi/build.rs if additional message types need serde serialization
• Verify tonic-build correctly processes the updated proto definitions
• Test that code generation produces the expected Rust types with new fields

🤖 Prompt for AI agents

Extend gRPC API to expose bus data.

Update proto submodule (git submodule update --init):
- Add transit_mode field to Station, Line, Company messages
- Add optional bus-specific fields: route_pattern to Line, pole_number to
Station
- Add/extend RPC methods for bus queries (get_bus_stops_by_route,
get_bus_routes_by_operator)
- Commit and push proto changes

Update grpc.rs controller:
- Implement new/extended bus-related RPC methods
- Add transit_mode filtering logic
- Default to railway data for backward compatibility
- Map domain entities with transit_mode to protobuf messages

Update build.rs to ensure proto compilation includes new fields

Phase 5: Implement Bus Data Synchronization Job

Create a background process that periodically fetches bus data from ODPT and updates the database.

Task 1: Create Synchronization Use Case

Implement the business logic for fetching and storing bus data.

• Create stationapi/src/use_case/interactor/sync_bus_data.rs module
• Implement sync_bus_data() function that orchestrates: fetching from ODPT client, transforming to domain entities, and persisting via repositories
• Add transaction handling to ensure atomic updates (all-or-nothing approach when updating bus data)
• Implement upsert logic to update existing bus records or insert new ones based on unique identifiers
• Add error handling and logging for sync failures
• Return sync statistics (records added, updated, errors encountered)

Task 2: Create Scheduled Background Task

Set up a periodic job runner within the application.

• Add tokio interval/timer support to schedule the sync job in stationapi/src/main.rs
• Create background task that calls sync_bus_data() at configurable intervals (e.g., daily)
• Read sync interval from environment variable or configuration file
• Ensure the task starts after application initialization but doesn't block server startup
• Add graceful shutdown handling for the background task
• Include health check endpoint that reports last successful sync timestamp

Task 3: Add Configuration for ODPT Integration

Provide configuration mechanism for ODPT API credentials and sync settings.

• Add environment variables for ODPT API access token (e.g., ODPT_API_TOKEN)
• Add environment variable for sync interval (e.g., BUS_SYNC_INTERVAL_HOURS, default to 24)
• Add environment variable to enable/disable bus sync (e.g., BUS_SYNC_ENABLED, default to true)
• Document required environment variables in project README or configuration guide
• Add validation at startup to ensure required ODPT configuration is present if sync is enabled

🤖 Prompt for AI agents

Implement background job to periodically sync bus data from ODPT.

Create sync_bus_data.rs use case:
- Orchestrate: fetch from ODPT client → transform to entities → persist via
repositories
- Use transactions for atomic updates (all-or-nothing)
- Implement upsert logic based on unique identifiers
- Add error handling, logging, and return sync statistics

Add background task in main.rs:
- Use tokio interval/timer for periodic execution
- Read sync interval from environment (BUS_SYNC_INTERVAL_HOURS, default 24)
- Start after initialization without blocking startup
- Add graceful shutdown handling
- Include health check endpoint with last sync timestamp

Add configuration:
- Environment variables: ODPT_API_TOKEN, BUS_SYNC_INTERVAL_HOURS,
BUS_SYNC_ENABLED
- Validate config at startup if sync enabled
- Document variables in README

Phase 6: Validation and Testing

Ensure data integrity for bus data and maintain existing validation patterns.

Task 1: Extend Data Validator for Bus Data

Update the validation tool to check bus data integrity.

• Update data_validator/src/main.rs to validate bus-specific records
• Add checks for bus stops referencing valid bus operators
• Add checks for bus routes referencing valid bus stops
• Validate that transit_mode values are consistent (e.g., bus stops only on bus routes)
• Follow the existing error accumulation pattern for reporting all validation issues
• Ensure validator can handle mixed train and bus data in the same tables

Task 2: Test ODPT Integration End-to-End

Verify the complete flow from ODPT API to gRPC response.

• Test ODPT client can successfully authenticate and fetch data
• Verify transformer correctly converts ODPT data to domain entities
• Confirm database contains bus data after sync job runs
• Test gRPC endpoints return bus data with correct transit_mode values
• Verify backward compatibility: existing train-only queries still work as expected
• Test filtering by transit_mode returns correct subsets

🤖 Prompt for AI agents

Add validation and end-to-end testing for bus data integration.

Extend data_validator/src/main.rs:
- Validate bus-specific records (stops reference valid operators, routes
reference valid stops)
- Check transit_mode consistency (bus stops only on bus routes)
- Follow existing error accumulation pattern
- Handle mixed train and bus data

Test end-to-end flow:
- ODPT client authentication and data fetching
- Transformer converts ODPT data to domain entities correctly
- Database contains bus data after sync
- gRPC endpoints return bus data with correct transit_mode
- Backward compatibility: train-only queries work as before
- Transit_mode filtering returns correct subsets

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Build the ODPT API integration infrastructure to fetch bus data.

Add reqwest, tokio, and serde_json dependencies to Cargo.toml

Create odpt_client.rs module with:
- Authentication using API token from environment
- Methods to fetch bus operators, routes (BusroutePattern), and stops
(BusStopPole)
- Error handling for network/auth failures and rate limits
- Retry logic with exponential backoff

Create odpt_models.rs with serde structs for:
- BusroutePattern (route info, operator reference, patterns)
- BusStopPole (location, name, pole number, operator references)
- Operator (company information)
- Include multilingual name fields (ja, en)
===============================================================================

Task: 2

Extend database schema to support both train and bus data.

Create migration script that:
- Adds transit_mode column (VARCHAR/ENUM: 'railway', 'bus') to companies, lines,
and stations tables
- Sets default value 'railway' for existing records
- Adds indexes on transit_mode columns
- Adds nullable bus-specific columns: route_pattern to lines, pole_number to
stations

Update CSV import in main.rs to:
- Explicitly set transit_mode='railway' when inserting railway data
- Maintain backward compatibility with existing CSV files
===============================================================================

Task: 3

Update domain model and repositories for multi-modal support.

Add transit_mode field to Station, Line, and Company entities
Add optional route_pattern to Line, add helper methods (is_bus, is_railway)

Update all repository implementations:
- Include transit_mode in all SQL queries (SELECT, INSERT, UPDATE)
- Add bus-specific methods: get_bus_stops_by_route, get_bus_routes_by_operator
- Default to filtering transit_mode='railway' for backward compatibility

Create odpt_transformer.rs to convert ODPT models to domain entities:
- BusStopPole → Station (transit_mode='bus')
- BusroutePattern → Line (transit_mode='bus')
- Operator → Company (transit_mode='bus')
- Handle coordinate transformation and multilingual field mapping
===============================================================================

Task: 4

Extend gRPC API to expose bus data.

Update proto submodule (git submodule update --init):
- Add transit_mode field to Station, Line, Company messages
- Add optional bus-specific fields: route_pattern to Line, pole_number to
Station
- Add/extend RPC methods for bus queries (get_bus_stops_by_route,
get_bus_routes_by_operator)
- Commit and push proto changes

Update grpc.rs controller:
- Implement new/extended bus-related RPC methods
- Add transit_mode filtering logic
- Default to railway data for backward compatibility
- Map domain entities with transit_mode to protobuf messages

Update build.rs to ensure proto compilation includes new fields
===============================================================================

Task: 5

Implement background job to periodically sync bus data from ODPT.

Create sync_bus_data.rs use case:
- Orchestrate: fetch from ODPT client → transform to entities → persist via
repositories
- Use transactions for atomic updates (all-or-nothing)
- Implement upsert logic based on unique identifiers
- Add error handling, logging, and return sync statistics

Add background task in main.rs:
- Use tokio interval/timer for periodic execution
- Read sync interval from environment (BUS_SYNC_INTERVAL_HOURS, default 24)
- Start after initialization without blocking startup
- Add graceful shutdown handling
- Include health check endpoint with last sync timestamp

Add configuration:
- Environment variables: ODPT_API_TOKEN, BUS_SYNC_INTERVAL_HOURS,
BUS_SYNC_ENABLED
- Validate config at startup if sync enabled
- Document variables in README
===============================================================================

Task: 6

Add validation and end-to-end testing for bus data integration.

Extend data_validator/src/main.rs:
- Validate bus-specific records (stops reference valid operators, routes
reference valid stops)
- Check transit_mode consistency (bus stops only on bus routes)
- Follow existing error accumulation pattern
- Handle mixed train and bus data

Test end-to-end flow:
- ODPT client authentication and data fetching
- Transformer converts ODPT data to domain entities correctly
- Database contains bus data after sync
- gRPC endpoints return bus data with correct transit_mode
- Backward compatibility: train-only queries work as before
- Transit_mode filtering returns correct subsets

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord or schedule a call!