Skip to content

Update Data model for MSP #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JakeTrevor merged 14 commits into from
Feb 6, 2026
Merged

Update Data model for MSP #356

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
29e2797
updated data model
JakeTrevor Jan 15, 2026
5493fbd
added mechanisms for custom weights and deadlines as well as the subm…
JakeTrevor Jan 22, 2026
3b34d0c
added architecture documentation
JakeTrevor Jan 22, 2026
f6673b6
Fixed some things for PR
JakeTrevor Feb 5, 2026
cec551c
added enums to snips
JakeTrevor Feb 5, 2026
f30d441
updated style guide with new enum stuff
JakeTrevor Feb 5, 2026
6bea1a6
minor tweak to schema
JakeTrevor Feb 5, 2026
3874851
fixed type errors
JakeTrevor Feb 5, 2026
7bf9c39
fixed lint errors
JakeTrevor Feb 5, 2026
260991e
Merge pull request #358 from amps-kt/jacobtrevor234/amp-231-update-da…
JakeTrevor Feb 5, 2026
31a0845
tiny fixup
JakeTrevor Feb 5, 2026
2fdfc73
Minor renamings in marking schema
JakeTrevor Feb 5, 2026
ca4a993
created migrations for new db schema
JakeTrevor Feb 5, 2026
9533e5a
Merge branch 'staging' into jacobtrevor234/amp-231-update-data-model
JakeTrevor Feb 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 20 additions & 9 deletions .vscode/snips.code-snippets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,18 @@
"scope": "javascript,typescript",
"prefix": "depr",
"body": ["/**", "* @deprecated $0", "*/"],
"description": "Log output to console",
"description": "Mark a feature as deprecated",
},

"Page": {
"prefix": "pasy",
"body": [
"export default async function Page() {",
" return (<>${0}</>);",
"}"
],
"description": "Creates a page server component with no params",
},
"prefix": "pasy",
"body": [
"export default async function Page() {",
" return (<>${0}</>);",
"}",
],
"description": "Creates a page server component with no params",
},

"Instance Page": {
"prefix": "inpag",
Expand Down Expand Up @@ -67,4 +67,15 @@
"export default ${0};",
],
},

"Enum": {
"prefix": "enm",
"body": [
"export const $1 = {",
"\t$2",
"} as const;",
"",
"export type $1 = keyof typeof $1;",
],
},
}
102 changes: 102 additions & 0 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
## Architecture

The SPA system is comprised of two main repositories:

The first is the matching service

This is a FastAPI (python) app which simply serves out a series of pre-defined algorithms.
All of the difficult logic being handled by libraries like networkx or matchingproblems.
As the name implies, it is used to handle matching problems, and only needs to be updated if there are new matching algorithms to add.

---

The second repository (this one) is a nextJS application which handles all the CRUD

We broadly follow the T3 stack and architecture;
more details can be found on [the t3 website](https://create.t3.gg/en/folder-structure-app?packages=prisma%2Ctailwind%2Ctrpc).
This link has the correct packages already selected.

The main components within this are:

- the database schema,

- We use [prisma](https://www.prisma.io/) as our ORM
- contained in `/prisma`

- the backend,

- We use [tRPC](https://trpc.io/)
- contained in `/src/router...`
- Most of the logic in the procedures in reality lives in data objects ([see below](#data-objects))

- the frontend,
- A set of [React](https://react.dev/) components, one for each page (see [NextJS](https://nextjs.org/))
- contained in `/src/app`
- This is the display layer; in general, very little business logic lives here.
Anything here can run on the client, so critical information needs to be controlled and restricted so

Some other critical dependencies:

- [Tailwind](https://tailwindcss.com/) for styling
- [Zod](https://tailwindcss.com/) for runtime data validation
- [ShadCN UI](https://ui.shadcn.com/) for many common UI tasks

Still, there are a few quirks unique to this application that we hand-rolled, which I will discuss in some more detail:

### Data objects

Previously, database calls were all done in the top level of procedures.
There were several issues with this - a big one being that it was difficult to re-use code.

This has been refactored, and we now use a system of data objects. These can be found in `/src/data-objects/`.

The idea is to organise functions by the objects they refer to; business logic related to a kind of object is encapsulated in the corresponding data object.

It's driven lots of very nice patterns; for instance, the authentication middleware we use is only possible because of this design.

In reality, the data objects are sometimes a bit messy; significant parts are not very well organised, and need cleaning up.
Maybe one day we will get around to it, but it hasn't been a priority.
We aim to remove all database calls from the tRPC procedures, and instead concentrate them in the data objects.

Generally, you will not need to import these directly; the relevant objects will be injected into the tRPC by the middleware.

### tRPC middlewares

We use a custom set of middlewares for tRPC.
These need to change very rarely.

The main idea is to abstract away a lot of the tedious boilerplate that is present in many procedures.

If you wanted, for example, to write a procedure which updates an allocation instance, then you would need to
accept the parameters which specify the instance (it's ID, essentially) and then query the database based on these.

Any procedure pertaining to an instance has to do this, and so we can save a lot of work by abstracting it away. This is what a middleware does. It adds the necessary input parameters and injects objects into the tRPC context.

We also have authentication middlewares - which automatically check that the user has the correct authorisation and inject a corresponding user object. If authentication fails, the procedure will error. Generally, you should try to make sure that your procedures are properly scoped wrt authorisation - it's critical to avoid accidental data leaks.

### Data transfer objects

There are many places in the application where _almost_ the same data is required, though it may differ slightly.
Creating types for each individual use case proved very messy; there were lots of interfaces floating around that were used only once,
and it made knowing what data was on each hard to track.
Now, instead, for each type of object there is a single type that contains all the data for that object type,
and we always pass around the full objects.

These large canonical types are called Data Transfer Objects (DTOs).

If you are presenting data relating to e.g. a student, you can use the `StudentDTO` type,
rather than having to carefully hand-craft the type for what you need.
Whilst this is a little less efficient - you may end up passing around data you don't strictly need - the benefits to code style and legibility are substantial.
It makes moving around different parts of the application much easier, since the types are always the same.

---

## Deployment

The matching service and the allocation app are each packaged into a docker image.
We use docker compose to orchestrate them, along with a few chron jobs for tasks that need to run regularly.
At some point in the future, we might move this into it's own container.

Some extra details need to be provided via environment variables.
Details of which ones are needed and what they should be can be found in `\src\env.ts`. Any variable with `.optional()` or `.default(_)` can be omitted.
`.env.example` contains an example env file.
6 changes: 0 additions & 6 deletions docs/dev/design.md
View file
Open in desktop

This file was deleted.

50 changes: 50 additions & 0 deletions docs/dev/style-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,53 @@ function QueryManager({ initialData }: { initialData: TData }) {
return <div>{data}</div>;
}
```

## Enums

When declaring an enum type, don't use magic strings or

Instead, use the following pattern:

```ts
export const MyEnum = {
OPTION_A: "OPTION_A",
OPTION_B: "OPTION_B",
OPTION_C: "OPTION_C",
// ... etc
} as const;

export type MyEnum = keyof typeof MyEnum;
```

- The dictionary and type names should both be in PascalCase and must match
- Enum values should be in `SCREAMING_SNAKE_CASE`
- The key and the value in the defining dict should match exactly.
- Export the inferred type on an additional line

There's a snippet in the repository for setting this up - the prefix is `enm`.

When referring to an enum value, use the dictionary

> Good:
>
> ```ts
> fun(MyEnum.OPTION_A);
> ```

> Bad:
>
> ```ts
> fun("OPTION_A");
> ```

When necessary, you should generate the related zod schema like so:

```ts
export const myEnumSchema = z.enum([
MyEnum.OPTION_A,
MyEnum.OPTION_B,
MyEnum.OPTION_C,
]);
```

- The name of the schema should be the name of the enum in camelCase with `Schema` appended to the end.
135 changes: 82 additions & 53 deletions prisma/schema/marking.prisma
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,91 +3,120 @@ enum MarkerType {
READER
}

model UnitOfAssessment {
id String @id @default(uuid())
flagId String @map("flag_id")
title String
open Boolean @default(false)
studentSubmissionDeadline DateTime @map("student_submission_deadline")
markerSubmissionDeadline DateTime @map("marker_submission_deadline")
weight Int
allowedMarkerTypes MarkerType[]
// ---
allocationGroupId String @map("allocation_group_id")
allocationSubGroupId String @map("allocation_sub_group_id")
allocationInstanceId String @map("allocation_instance_id")
// ---
flag Flag @relation(fields: [flagId, allocationGroupId, allocationSubGroupId, allocationInstanceId], references: [id, allocationGroupId, allocationSubGroupId, allocationInstanceId], onDelete: Cascade, map: "unit_flag")
allocationInstance AllocationInstance @relation(fields: [allocationGroupId, allocationSubGroupId, allocationInstanceId], references: [allocationGroupId, allocationSubGroupId, id], onDelete: Cascade, map: "unit_instance")
assessmentCriteria AssessmentCriterion[]
markerSubmissions MarkingSubmission[]
finalUnitOfAssessmentGrades FinalUnitOfAssessmentGrade[]
enum MarkingStatus {
PENDING
DONE
MODERATE
NEGOTIATE
}

@@map("unit_of_assessment")
enum MarkingMethod {
AUTO
OVERRIDE
NEGOTIATED
MODERATED
}

model AssessmentCriterion {
id String @id @default(uuid())
unitOfAssessmentId String @map("unit_of_assessment_id")
enum FinalGradeMethod {
AUTO
OVERRIDE
}

model MarkingComponent {
id String @id @default(uuid())
unitOfAssessmentId String @map("unit_of_assessment_id")
title String
description String
weight Int
layoutIndex Int @map("layout_index")
layoutIndex Int @map("layout_index")
// ---
unitOfAssessment UnitOfAssessment @relation(fields: [unitOfAssessmentId], references: [id], onDelete: Cascade, map: "criterion_unit")
scores CriterionScore[]
unitOfAssessment UnitOfAssessment @relation(fields: [unitOfAssessmentId], references: [id], onDelete: Cascade, map: "marking_component_uoa")
scores MarkingComponentSubmission[]

@@unique([title, unitOfAssessmentId])
@@map("assessment_criterion")
@@map("marking_component")
}

model MarkingSubmission {
summary String
grade Int
recommendedForPrize Boolean @default(false) @map("recommended_for_prize")
draft Boolean
markerId String @map("marker_id")
studentId String @map("student_id")
unitOfAssessmentId String @map("unit_of_assessment_id")
model MarkingComponentSubmission {
markerId String @map("marker_id") // marker type?
studentId String @map("student_id")
unitOfAssessmentId String @map("unit_of_assessment_id")
grade Int
justification String
// ---
unitOfAssessment UnitOfAssessment @relation(fields: [unitOfAssessmentId], references: [id], onDelete: Cascade, map: "submission_unit")
criterionScores CriterionScore[]
markingComponentId String @map("marking_component_id")
// ---
markingComponent MarkingComponent @relation(fields: [markingComponentId], references: [id], onDelete: Cascade, map: "marking_component_submission_marking_component")
submission UnitOfAssessmentSubmission @relation(fields: [markerId, studentId, unitOfAssessmentId], references: [markerId, studentId, unitOfAssessmentId], onDelete: Cascade, map: "marking_component_submission_uoa_submission")

@@id([markerId, studentId, unitOfAssessmentId], name: "studentMarkerSubmission")
@@map("assessment_marking_submission")
@@id([markerId, studentId, markingComponentId], name: "markingComponentSubmission")
@@map("marking_component_submission")
}

model CriterionScore {
markerId String @map("marker_id")
studentId String @map("student_id")
unitOfAssessmentId String @map("unit_of_assessment_id")
grade Int
justification String
model UnitOfAssessment {
id String @id @default(uuid())
flagId String @map("flag_id")
title String
open Boolean @default(false)
defaultStudentSubmissionDeadline DateTime @map("default_student_submission_deadline")
// Days to mark
markerSubmissionDeadline DateTime @map("marker_submission_deadline")
defaultWeight Int @map("default_weight")
allowedMarkerTypes MarkerType[]
// ---
assessmentCriterionId String @map("assessment_component_id")
allocationGroupId String @map("allocation_group_id")
allocationSubGroupId String @map("allocation_sub_group_id")
allocationInstanceId String @map("allocation_instance_id")
// ---
criterion AssessmentCriterion @relation(fields: [assessmentCriterionId], references: [id], onDelete: Cascade, map: "score_criterion")
submission MarkingSubmission @relation(fields: [markerId, studentId, unitOfAssessmentId], references: [markerId, studentId, unitOfAssessmentId], onDelete: Cascade, map: "score_submission")
flag Flag @relation(fields: [flagId, allocationGroupId, allocationSubGroupId, allocationInstanceId], references: [id, allocationGroupId, allocationSubGroupId, allocationInstanceId], onDelete: Cascade, map: "unit_flag")
allocationInstance AllocationInstance @relation(fields: [allocationGroupId, allocationSubGroupId, allocationInstanceId], references: [allocationGroupId, allocationSubGroupId, id], onDelete: Cascade, map: "uoa_instance")
markingComponents MarkingComponent[]
markerSubmissions UnitOfAssessmentSubmission[]
grades UnitOfAssessmentGrade[]

@@id([markerId, studentId, assessmentCriterionId], name: "markingCriterionSubmission")
@@map("criterion_score")
@@map("unit_of_assessment")
}

model FinalUnitOfAssessmentGrade {
model UnitOfAssessmentSubmission {
summary String
grade Int
recommendedForPrize Boolean @default(false) @map("recommended_for_prize")
draft Boolean
markerId String @map("marker_id")
studentId String @map("student_id")
unitOfAssessmentId String @map("unit_of_assessment_id")
// ---
unitOfAssessment UnitOfAssessment @relation(fields: [unitOfAssessmentId], references: [id], onDelete: Cascade, map: "uoa_submission_uoa")
criterionScores MarkingComponentSubmission[]

@@id([markerId, studentId, unitOfAssessmentId], name: "uoaSubmissionId")
@@map("unit_of_assessment_submission")
}

model UnitOfAssessmentGrade {
unitOfAssessmentId String @map("unit_of_assessment_id")
unitOfAssessment UnitOfAssessment @relation(fields: [unitOfAssessmentId], references: [id], onDelete: Cascade, map: "final_unit_grade_unit")
studentId String @map("student_id")
// ---
grade Int
comment String
status MarkingStatus @default(PENDING)
method MarkingMethod @default(AUTO)
//
submitted Boolean @default(false)
customDueDate DateTime? @map("custom_due_date")
customWeight Int? @map("custom_weight")

@@id([studentId, unitOfAssessmentId], name: "studentAssessmentGrade")
@@map("final_unit_of_assessment_grade")
@@id([studentId, unitOfAssessmentId], name: "uoaGradeId")
@@map("unit_of_assessment_grade")
}

model FinalGrade {
id String @id @default(uuid())
studentId String @map("student_id")
grade Int
comment String @default("")
method FinalGradeMethod @default(AUTO)
// ---
allocationGroupId String @map("allocation_group_id")
allocationSubGroupId String @map("allocation_sub_group_id")
Expand Down
Loading
Loading