IRECKONU Assignment

Requirements

Build a web app with the features described below.

Backend - C# .NET Core web API

1. Import the file via web API, expect the file to be very large in production

2. Transform the data into a logical model

3. Store the data (two locations - a Database and JSON file)

o Database (can be either MS SQL or MongoDB - you choose)

o JSON file on the disk

Data: Sample File: https://goo.gl/tJWo1f

Criteria: Your implementation properly follows all SOLID principles.

Submitted Task Branch : SubmittedTask

Solution

1- ASP.NET Core 3.1 API to upload the files

2-.NET Core 3.1 console application + Hangfire + Topshelf to process the files in the background or on a seperate server

Main Inspirations

1- Adaptive Code: Agile coding with design patterns and SOLID principles (2nd Edition) (Developer Best Practices) Chapter (11) - Stairway Pattern

2- Avoid Referencing Infrastructure in Visual Studio Solutions

3- Modular Monolith with DDD

4- Dependency Injection Principles, Practices, and Patterns

5- Domain-Driven-Design-Example C#

Tools

• ASP.NET Core 3.1
• .NET Core 3.1
• SQL Server
• Autofac
• Hangfire
• Topshelf (Could have been replaced with service worker )
• TinyCsvParser (choosed based on this Link)
• Distributed Caching (In-memory)

Possible Enhancements

• Write Unit Tests
• Eliminate SRP Violations
• Reconsider Database Context lifetime in Hangefire application
• Reconsider Serializable attribute on domain models
• Docker (Compose)
• Enhance Dynamic Assembly loading
• Finish Caching Save Decorator
• TinyCsvParser parallel Processing
• Parallel Processing
• Review Naming Conventions
• Add Redis as caching server and as message queue
• Remove Hangfire and go completely Event Driven (consider outbox )
• Save files to azure using azure storage

How to Run

Note: Admin Privilege is required (to save files)

1- Run Update-Database using project: IReckonu.DataImportingTool.Data.SqlServer while using IReckonu.DataImportingTool.ProcessingApplication as your startup project (will fails if API project is the startup project due to dynamic assembly loading)

2- Run IReckonu.DataImportingTool.ProcessingApplication

3- Run IReckonu.DataImportingTool.API