Setting Up and Running the Mapping Service

This guide outlines the steps to set up and run the mapping service application.

Prerequisites

• Node.js: Version 20.0 or higher. You can check your Node.js version by running node -v in your terminal.
• Docker: Ensure Docker is installed and running on your system.

How to Use the App

1. Create Environment Configuration: First, create a .env file in the root directory of the project. Populate this file with the following values:

   # ==== Application Port ====
   NODE_SERVER_PORT=6600
   
   # ==== Node environment ====
   NODE_ENV=development
   
   # ==== Redis configurations ====
   REDIS_URL='redis://localhost:6379'
   
   # ==== DB URL ======
   DATABASE_URL="postgresql://postgres:postgres@localhost:5440/mentalyc_db?schema=public"
   
   JWT_SECRET='secret'

2. Spin Up Docker Containers: Run the following npm script to start the PostgreSQL and Redis services using Docker:

   npm run db:up

3. Run Database Migrations: Execute the migrations to set up the database schema:

   npm run migrate

4. Seed the Database: Populate the database with initial user and places data using the seed script:

   npm run seed

5. Start the Development Environment: Run the following command to start the application in development mode:

   npm run dev

   The API will be accessible at http://localhost:6600/. Metrics will be available at http://localhost:8080/.

   For API documentation, navigate to http://localhost:6600/api-docs.

How To Run Tests

To execute the tests for this project, please follow these steps:

1. Spin Up Test Databases: Run the following npm script to start the necessary test database instances:

   npm run db:up

2. Run Test Migrations: Execute the database migrations specifically for the testing environment:

   npm run db:test:migrate

3. Run Tests: Finally, execute the test suite using the following npm script:

   npm run test

Available API Routes

The API provides the following routes:

• Add Place: Endpoint to create a new place.
• Update Place: Endpoint to modify an existing place's details.
• Get Place by ID: Endpoint to retrieve information about a specific place using its unique identifier.
• Get All Places (with search queries): Endpoint to retrieve a list of all places, with optional query parameters for searching and filtering.
• Get Places within a Geographical Radius: Endpoint to find places within a specified distance from a given coordinate.
• Get Navigation Route: Endpoint to retrieve a route between a specified origin place and a destination place.

---

1. Data Model and Storage Setup

For storing location and spatial data, I choose PostgreSQL with the PostGIS extension. Here's why:

• Robustness and Reliability: PostgreSQL is a mature and highly reliable open-source relational database.
• Spatial Data Expertise: PostGIS provides powerful spatial data types and functions for efficient storage and querying of geographic data.
• Scalability: PostgreSQL can be scaled both vertically (increasing resources on a single server) and horizontally (using techniques like read replicas and sharding).
• ACID Compliance: Ensures data integrity and consistency, crucial for location data.

Spatial Indexing:

A GiST (Generalized Search Tree) index on the location column in the places table is crucial for efficient geolocation-based queries. This allows PostgreSQL/PostGIS to quickly identify places within a certain distance or bounding box.

2. Services to Serve the Features

This backend was built using Node.js with TypeScript and Express.js. This offers:

• Performance: Node.js's non-blocking I/O model is well-suited for handling concurrent requests.
• Scalability: Node.js applications can be easily scaled horizontally using process managers like PM2 or containerization with Docker and orchestration with Kubernetes.
• Developer Productivity: TypeScript adds static typing, improving code maintainability and reducing errors. Express.js is a minimalist and flexible web framework.
• JavaScript Ecosystem: A large and active community with a wide range of libraries.

API Endpoints:

These are the implemented RESTful API endpoints:

• POST /places: Create a new place. Request body would include name, latitude, longitude, category, address, and description.
• GET /places/:id: Retrieve details of a specific place by its ID.
• GET /places/nearby?lat={latitude}&lon={longitude}&radius={distance_in_meters}: Retrieve places within a specified radius of a given coordinate. This would leverage PostGIS's ST_DWithin function.
• GET /places?q={query}: Search for places by name or description (using PostgreSQL's ILIKE or full-text search capabilities).
• GET /route?origin_id={origin_place_id}&destination_id={destination_place_id}: Retrieve a route between two places. For the basic prototype, this could return a simple ordered list of place IDs or coordinates. A more advanced implementation would involve integrating with a routing engine.

Algorithms:

• Geolocation-based queries: I utilized PostGIS's spatial functions like ST_DWithin to find places within a specified distance. The query looks like:

  SELECT id, name, latitude, longitude
  FROM places
  WHERE ST_DWithin(location, ST_SetSRID(ST_MakePoint($1, $2), 4326), $3);

  where $1 is longitude, $2 is latitude, and $3 is the radius in meters.

• Basic route search: For the basic prototype, I only returned the coordinates for the origin and destination location and the distance between them. A more advanced implementation would require a graph representation of the road network and algorithms like Dijkstra's or A*.

3. Performance Optimization and Observability

Performance Optimization:

• Database Indexing: I implemented appropriate indexes on frequently queried columns (name, category) and, most importantly, the spatial index on the location column.
• Connection Pooling: I used Prisma Client which uses a connection pool under the hood to efficiently manage database connections and reduce overhead.
• Data Caching: I implemented caching mechanisms using, Redis for frequently accessed place data or the results of nearby place queries.
• Pagination: For queries that might return a large number of results (e.g., search), I implemented pagination to improve response times and reduce resource usage.
• Asynchronous Operations: I utilized Node.js's asynchronous capabilities to handle I/O-bound operations efficiently.

Observability:

• Logging: Implement comprehensive logging using a library like Winston or Bunyan to track application behavior, errors, and performance. Structure logs for easy searching and analysis.
• Metrics: Integrate a metrics collection library like Prometheus and a visualization tool like Grafana to monitor key performance indicators (KPIs) such as:
  • Request latency
  • Request throughput
  • Error rates
  • Database query performance
  • Resource utilization (CPU, memory)
• Tracing: Implement distributed tracing using tools like Jaeger or Zipkin to track requests across different services and identify performance bottlenecks.
• Health Checks: Implement health check endpoints (e.g., /health) that report the status of the application and its dependencies (database). This is crucial for monitoring and automated scaling.

4. Testing the Software on the Go

Local Development and Testing:

• Docker Compose: Use Docker Compose to define and manage the PostgreSQL database and the Node.js backend as containers. This provides a consistent and reproducible development environment.
• Seed Data: Include a script to seed the database with sample place data for testing purposes.
• API Testing Tools: Use tools like Postman, Insomnia, or curl to manually test the API endpoints.

🌟 Bonus Scope

1. Further Performance Improvement

• Routing Engine Integration: For more advanced route search, I will integrate a dedicated routing engine like Open Source Routing Machine (OSRM) or Valhalla. These engines use graph representations of road networks and sophisticated algorithms to calculate optimal routes, and the backend will make API calls to these services.
• Geocoding Service: I will integrate a geocoding service (e.g., Nominatim) to convert addresses into coordinates and vice versa. This will enhance the place creation and search functionalities.
• Read Replicas: For high read loads, I will set up read replicas of the PostgreSQL database. The application will direct read queries to these replicas, reducing the load on the primary database.
• Database Sharding: For extreme scale, I will consider sharding the database based on geographic regions or other criteria to distribute the data and query load across multiple database instances.
• Load Balancing: I will use a load balancer (e.g., Nginx, HAProxy) to distribute incoming traffic across multiple instances of the backend application.
• Content Delivery Network (CDN): If the application serves static map tiles or other static assets in the future, I will use a CDN to significantly improve performance for users across the globe.

Dealing with Differing Usage Patterns Worldwide:

• Regional Data Partitioning: If usage is heavily concentrated in specific regions, I will consider partitioning the place data geographically. This could involve separate database clusters or more sophisticated sharding strategies.
• Caching Strategies: I will implement region-specific caching strategies based on expected usage patterns. For example, frequently accessed places in high-usage areas could have longer cache durations.
• Auto-scaling: I will utilize auto-scaling capabilities provided by cloud platforms (e.g., AWS Auto Scaling, Google Cloud Autoscaler) to dynamically adjust the number of backend instances based on traffic load in different regions.

Worldwide Availability

• Multi-Region Deployment: I will deploy the backend application and database across multiple geographic regions using cloud provider services. This ensures low latency for users worldwide and provides redundancy in case of regional outages.
• Global Load Balancing: I will use a global load balancer (e.g., AWS Global Accelerator, Google Cloud Load Balancing) to route user requests to the nearest available region, minimizing latency.
• Data Replication: I will implement database replication across regions to ensure data consistency and availability. I will consider asynchronous replication for better write performance or synchronous replication for stronger consistency guarantees (with potential latency trade-offs).
• Edge Caching: I will utilize edge caching through CDNs to cache static and potentially even dynamic content closer to users globally.

. End-to-End Provision

• Infrastructure as Code (IaC): I will use IaC tools like Terraform or CloudFormation to define and manage the entire infrastructure (database, backend instances, load balancers, etc.) in code. This allows for automated provisioning, version control, and repeatable deployments.
• Containerization: I will package the backend application into Docker containers to ensure consistent deployment across different environments.
• Orchestration: I will use a container orchestration platform like Kubernetes to manage the deployment, scaling, and health of the backend containers across multiple nodes and regions.
• CI/CD Pipeline: I will implement a comprehensive CI/CD pipeline that automatically builds, tests, and deploys code changes to the different environments (development, staging, production). This pipeline will integrate with the IaC tools for infrastructure updates.

Security

• HTTPS: I will enforce HTTPS for all API communication to encrypt data in transit.
• Authentication: I will implement a robust authentication mechanism (e.g., JWT-based authentication) to verify the identity of clients accessing the API.
• Authorization: I will implement fine-grained authorization to control what actions authenticated users are allowed to perform (e.g., using RBAC - Role-Based Access Control).
• Input Validation: I will thoroughly validate all incoming data to prevent injection attacks (e.g., SQL injection, XSS). I will use libraries specifically designed for input validation.
• Rate Limiting and Throttling: I will implement rate limiting and throttling to prevent abuse and denial-of-service (DoS) attacks.
• CORS (Cross-Origin Resource Sharing): I will configure CORS headers to restrict which domains can access the API.
• Database Security: I will secure the database with strong passwords, network firewalls, and regular security updates. I will follow the principle of least privilege for database user accounts.
• Secrets Management: I will use secure secrets management tools (e.g., HashiCorp Vault, AWS Secrets Manager) to store and manage sensitive information like database credentials and API keys. I will avoid hardcoding secrets in the application code.
• Network Security: I will implement network security measures like firewalls and network segmentation to control traffic flow and isolate different parts of the infrastructure.
• Regular Security Audits: I will conduct regular security audits and penetration testing to identify and address potential vulnerabilities.
• Dependency Management: I will keep all dependencies (libraries, frameworks) up-to-date with the latest security patches. I will use tools to scan for known vulnerabilities in dependencies.
• Monitoring and Alerting: I will set up monitoring and alerting for security-related events (e.g., unusual API activity, failed login attempts).