Skip to content

Generate types from SQL database schema#35

Open
nooreldeensalah wants to merge 7 commits intomainfrom
sql-ts
Open

Generate types from SQL database schema#35
nooreldeensalah wants to merge 7 commits intomainfrom
sql-ts

Conversation

@nooreldeensalah
Copy link
Contributor

@nooreldeensalah nooreldeensalah commented Feb 24, 2025

Closes #22

TODO:

  • Also check in the CI that the interface is up-to-date with sql definitions.

Summary by CodeRabbit

  • Refactor
    • Streamlined internal data structure definitions for improved consistency and maintainability by centralizing type declarations.
  • New Features
    • Integrated enhanced SQLite support with updated dependencies.
    • Introduced a new utility for generating type definitions from the database schema.
  • Chores
    • Strengthened database integrity by enforcing mandatory fields in key data tables.
  • Bug Fixes
    • Improved error handling in the Narrator page to prevent undefined data issues.
    • Enhanced robustness of components by providing default messages for missing data.
    • Prevented runtime errors by safely handling undefined identifiers in graph and chain visualizations.

@coderabbitai
Copy link

coderabbitai bot commented Feb 24, 2025

Walkthrough

This pull request restructures the interface declarations in lib/sqlite.ts by replacing custom definitions with extensions of types imported from ../types/database. The changes update the database schema in scripts/gen_db.py by adding NOT NULL constraints, introduce a new configuration file (sql-ts.config.json), and add a new TypeScript file (types/database.ts) that defines key interfaces. Additionally, package.json now includes a new script for type generation and updated dependencies for SQLite support.

Changes

File(s) Change Summary
lib/sqlite.ts Replaced custom interfaces (Hadith, Narrator, InfoSource, Chain) with extensions of imported types (Hadiths, Rawis, Sources, HadithChains).
package.json Added new script "generate-types": "bunx @rmp135/sql-ts -c sql-ts.config.json", dependency "sqlite3": "^5.1.7", and devDependency "@rmp135/sql-ts": "^2.2.0".
scripts/gen_db.py Updated create_tables function by adding NOT NULL constraints to critical columns across hadiths, rawis, hadith_chains, and sources tables; filtered null scholar indices in inserts.
sql-ts.config.json New configuration file specifying settings for a SQLite client, including connection details, casing for tables, interface naming, and Prettier formatting with output path ./types/database.
types/database.ts New file defining interfaces HadithChains, Hadiths, Rawis, and Sources that structure the Hadith-related data.

Sequence Diagram(s)

sequenceDiagram
    participant Dev as Developer
    participant Script as Generate-Types Script
    participant Tool as @rmp135/sql-ts Tool
    participant Config as sql-ts.config.json
    participant DB as SQLite Database
    participant Types as types/database.ts

    Dev->>Script: Execute "generate-types" command
    Script->>Tool: Run type generation
    Tool->>Config: Read configuration details
    Tool->>DB: Connect using provided database file
    Tool->>Types: Generate/update TypeScript interfaces
    note over Types: Interfaces are updated for use in lib/sqlite.ts
Loading

Assessment against linked issues

Objective Addressed Explanation
Document nullable fields in the DB schema and auto generate TypeScript schema from it (#22)

Possibly related PRs

  • Add chapter page #33: Refactors type definitions in lib/sqlite.ts by adding a new Chapter interface and updating functions to use chapter as a string instead of chapterNo as a number; related due to overlapping changes in type declarations and hadith chapter handling.

Poem

Oh, I hop through lines of code so neat,
Interfaces now streamlined—what a treat!
Tables stand strong with constraints so tight,
Types are generated with pure delight.
My fluffy ears twitch to the new script’s tune,
Celebrating progress under the coding moon.
🐇 Hoppity, hoppity—let the changes bloom!

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

app/narrator/[name]/page.tsx

Oops! Something went wrong! :(

ESLint: 9.25.1

ESLint couldn't find an eslint.config.(js|mjs|cjs) file.

From ESLint v9.0.0, the default configuration file is now eslint.config.js.
If you are using a .eslintrc.* file, please follow the migration guide
to update your configuration file to the new format:

https://eslint.org/docs/latest/use/configure/migration-guide

If you still have problems after following the migration guide, please stop by
https://eslint.org/chat/help to chat with the team.


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 70b6438 and 5f188ea.

📒 Files selected for processing (1)
  • app/narrator/[name]/page.tsx (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/narrator/[name]/page.tsx
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: deploy

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
scripts/gen_db.py (1)

291-295: Address the FIXME comment regarding data quality.

The comment indicates potential data quality issues with duplicates. Would you like me to help investigate the root cause and propose a solution?

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6360fc9 and c153ff9.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (5)
  • lib/sqlite.ts (1 hunks)
  • package.json (2 hunks)
  • scripts/gen_db.py (3 hunks)
  • sql-ts.config.json (1 hunks)
  • types/database.ts (1 hunks)
✅ Files skipped from review due to trivial changes (2)
  • sql-ts.config.json
  • types/database.ts
🧰 Additional context used
🪛 GitHub Actions: Deployment
lib/sqlite.ts

[error] 4-7: An interface declaring no members is equivalent to its supertype. @typescript-eslint/no-empty-object-type

🔇 Additional comments (4)
lib/sqlite.ts (1)

49-150: SQL statements look secure and well-optimized.

The SQL statements are properly parameterized to prevent SQL injection, and the LEFT JOIN operations are correctly used for optional relationships. The statements align well with the indexes created in the database schema.

scripts/gen_db.py (1)

11-16: Well-structured NOT NULL constraints.

The addition of NOT NULL constraints to key columns improves data integrity and ensures that the generated TypeScript types accurately reflect the required fields in the database schema.

Also applies to: 24-27, 37-39, 48-51

package.json (2)

10-10: Review sql-ts.config.json for type generation settings.

The new script references sql-ts.config.json. Please provide this configuration file for review to ensure proper type generation settings.


69-69:

Details

❓ Verification inconclusive

Verify the compatibility of sqlite3 and sql-ts versions.

Let's check if these are the latest stable versions and if they have any known compatibility issues.

Also applies to: 76-76


🌐 Web query:

What is the latest version of sqlite3 for Node.js?

💡 Result:

As of February 2025, there are two primary options for SQLite integration in Node.js:

Built-in Experimental Module

Node.js v22.5+ includes experimental native SQLITE support through node:sqlite, though it requires using --experimental-sqlite flag during execution[1][7]. This provides basic synchronous operations but lacks some advanced features found in third-party libraries[7].

Third-Party Libraries

Better-SQLITE3

Latest Version: v11.8. (Jan 2025)[6][10]
Key features:

  • Synchronous API with transaction support
  • Worker thread integration
  • Benchmark performance up to 24x faster than alternatives[6][10]
npm install better-sqlite3@latest

Node-SQLITE Wrapper

Latest Version: v5. (Nov 2024)[4][5]
A TypeScript wrapper with:

  • ES6 promise support
  • SQL migration tools
npm install sqlite@latest
Library Performance API Style Maintenance
Built-in Module Moderate Experimental Active development
Better-SQLITE3 High Synchronous Active (v11.x)
Node-SQLITE Medium Promise-based Maintained

For most production use cases requiring maximum performance and stability, Better-SQLITE3 v11.x remains recommended despite Node.js' new native implementation still being experimental[6][7][10]. Developers needing promise-based workflows might prefer node-sqlite wrapper[4][5].

Citations:


Dependency Versions Compatibility Check

  • The "sqlite3": "^5.1.7" dependency appears to match the current stable release for Node.js-based SQLite integration. Based on the recent web query, there are no known compatibility issues with this version.
  • For the newly added sql-ts (also referenced at line 76), please ensure that its version and integration align with your type generation requirements and work smoothly alongside sqlite3. Although no conflicts are evident from available documentation, it’s worthwhile to double-check that both dependencies continue to function as expected.

Please verify this integration during your testing phase, especially if you plan future upgrades or consider alternative libraries (e.g., better-sqlite3) for performance improvements.

lib/sqlite.ts Outdated
Comment on lines 4 to 7
export interface Hadith extends Hadiths {}
export interface Narrator extends Rawis {}
export interface InfoSource extends Sources {}
export interface Chain extends HadithChains {}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Consider alternatives to empty interfaces.

The empty interfaces violate the TypeScript linting rule @typescript-eslint/no-empty-object-type. Consider these alternatives:

  1. Use type aliases instead of interfaces:
-export interface Hadith extends Hadiths {}
-export interface Narrator extends Rawis {}
-export interface InfoSource extends Sources {}
-export interface Chain extends HadithChains {}
+export type Hadith = Hadiths;
+export type Narrator = Rawis;
+export type InfoSource = Sources;
+export type Chain = HadithChains;
  1. Or add properties to the interfaces if they need to be extended in the future.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export interface Hadith extends Hadiths {}
export interface Narrator extends Rawis {}
export interface InfoSource extends Sources {}
export interface Chain extends HadithChains {}
export type Hadith = Hadiths;
export type Narrator = Rawis;
export type InfoSource = Sources;
export type Chain = HadithChains;
🧰 Tools
🪛 GitHub Actions: Deployment

[error] 4-7: An interface declaring no members is equivalent to its supertype. @typescript-eslint/no-empty-object-type

@github-actions
Copy link

Cloudflare Pages Deployment

Environment: preview
Project: isnad
Built with commit: 8cb5f2f
Preview URL: https://d243e14b.isnad-acg.pages.dev
Branch Preview URL: https://sql-ts.isnad-acg.pages.dev

Wrangler Output

Uploading... (31/4926)
Uploading... (86/4926)
Uploading... (145/4926)
Uploading... (225/4926)
Uploading... (770/4926)
Uploading... (1069/4926)
Uploading... (1836/4926)
Uploading... (1950/4926)
Uploading... (2926/4926)
Uploading... (4926/4926)
✨ Success! Uploaded 4895 files (31 already uploaded) (17.52 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://d243e14b.isnad-acg.pages.dev

Comment on lines +163 to +165
if (narrator.scholar_indx === undefined) {
notFound();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I marked scholar_indx as not null and in the gen_db.py script database screamed back at me

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to filter them out. The scholar index is used in the transmission chain. If the index is null they won't appear in any chain.

Comment on lines +33 to +34
scholar_indx?: number;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is it still nullable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the library arbitrarily assigns ? to the fields, it removed the | null when I added NOT NULL, refer to the last commit to see the changes

@ElGarash ElGarash deleted a comment from akbany99 Feb 26, 2025
@nooreldeensalah nooreldeensalah marked this pull request as ready for review February 26, 2025 19:47
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
components/predecessors-successors-chain.tsx (1)

112-114: Type safety concern in viewGenerator function.

The generic type for nodeData has been removed and replaced with a type assertion. While this works, it reduces compile-time type safety.

Consider explicitly typing the parameter to maintain type safety:

-            viewGenerator: (nodeData) => (
-              <NarratorCard {...(nodeData as NarratorGraphNode)} />
+            viewGenerator: (nodeData: NarratorGraphNode) => (
+              <NarratorCard {...nodeData} />
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c153ff9 and ba13997.

📒 Files selected for processing (10)
  • app/hadith/[source]/[chapter]/[hadithNo]/page.tsx (1 hunks)
  • app/narrator/[name]/page.tsx (1 hunks)
  • components/narrator-card.tsx (1 hunks)
  • components/predecessors-successors-chain.tsx (4 hunks)
  • components/transmission-chain.tsx (4 hunks)
  • lib/sqlite.ts (4 hunks)
  • lib/types/graph.ts (1 hunks)
  • scripts/gen_db.py (3 hunks)
  • tests/sqlite.test.ts (2 hunks)
  • types/database.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • types/database.ts
  • scripts/gen_db.py
🔇 Additional comments (26)
app/narrator/[name]/page.tsx (1)

163-165: Good addition to handle undefined scholar indices.

This check ensures that narrator pages with undefined scholar_indx values won't proceed to fetch related data, preventing potential runtime errors. The use of notFound() appropriately signals a 404 response when the scholar index is undefined.

lib/types/graph.ts (1)

17-19: Clean inheritance refinement for HadithGraphNode interface.

The updated inheritance structure properly extends Omit<HadithWithChain, "id"> and BaseGraphNode, avoiding property collision by excluding the id property from HadithWithChain while inheriting it from BaseGraphNode. This aligns with the broader type system reorganization.

components/narrator-card.tsx (1)

7-13: Improved type flexibility for formatDate function.

The function now accepts a wider range of input types (string | number | null | undefined), making it more robust. The loose equality check (==) efficiently handles both null and undefined cases with a single comparison.

app/hadith/[source]/[chapter]/[hadithNo]/page.tsx (1)

62-65: Good use of nullish coalescing for fallback content.

Adding fallback text when hadith.text_ar or hadith.explanation are null/undefined improves the user experience by displaying meaningful content instead of empty fields. This defensive programming approach aligns with the enhanced type safety goals of this PR.

components/transmission-chain.tsx (5)

29-29: Type signature updated for better typing.

The type signature has been updated from HadithWithChain to HadithGraphNode, which correctly aligns with both the interface definition in lib/types/graph.ts and how this function is used in the component.


56-60: Good defensive programming with null handling.

The addition of nullish coalescing operators (?? 0) ensures the code handles cases where scholar_indx might be null or undefined, preventing potential runtime errors.


68-74: Consistent null-safety handling for scholar_indx.

The nullish coalescing operator (?? 0) ensures consistent behavior when working with potentially undefined scholar_indx values. This pattern is correctly applied throughout the component.


94-95: Defensive programming for node ID assignment.

The nullish coalescing operator ensures that even if scholar_indx is null or undefined, the node will still have a valid string ID.


105-107: Null safety in graph link creation.

This defensive programming approach ensures that the graph links are always created with valid string values for source and target, preventing potential visualization errors.

tests/sqlite.test.ts (2)

88-88: Property name updated to match database schema.

The property has been changed from id to hadith_id to match the field name in the database schema, ensuring correct comparison between hadiths.


97-99: Improved robustness of ID comparison.

The updated code adds defensive programming by:

  1. Converting IDs to strings (with fallback to "0")
  2. Explicitly parsing as integers for comparison

This prevents potential errors when IDs might be undefined or not in the expected format.

components/predecessors-successors-chain.tsx (5)

43-44: Added null safety for scholar_indx in ID generation.

The addition of optional chaining (?.toString()) with a fallback empty string ensures that valid IDs are generated even when scholar_indx is null or undefined.


55-57: Null-safety in graph link creation.

The optional chaining and nullish coalescing ensures that source and target values are always valid strings, preventing potential errors in the graph visualization.


64-65: Improved handling for central node ID.

Adding optional chaining with a fallback to "unknown" ensures the central node always has a valid ID, even if scholar_indx is null or undefined.


71-72: Consistent null handling for student nodes.

This follows the same pattern as for other nodes, ensuring consistent ID generation throughout the graph construction.


83-85: Defensive programming in link creation.

The addition of optional chaining and nullish coalescing for both source and target IDs prevents potential graph rendering errors.

lib/sqlite.ts (10)

1-10: Clean type refactoring with proper re-exports.

The replacement of custom interface definitions with imports from a dedicated types file is an excellent improvement that:

  1. Centralizes type definitions
  2. Follows the "single source of truth" principle
  3. Makes generated types from the database schema directly usable

The re-exports maintain backward compatibility with existing code, allowing for a smooth transition.


14-15: Updated composite types to use imported base types.

The composite types now use the imported base types, maintaining consistency with the new type system.


158-161: Return type updated to match database schema.

The return type has been updated from the custom Hadith interface to the generated Hadiths type, ensuring accurate representation of the database schema.


163-166: Return type updated for getNarrators.

The function now correctly returns Rawis[] instead of the custom Narrator[], which aligns with the database schema.


168-179: Updated return type for getHadithById.

The function now correctly returns Hadiths | null instead of the custom Hadith | null.


194-197: Return type updated for getAllHadiths.

The function now returns Hadiths[] which directly matches the database schema.


210-215: Updated return type for getNarrator function.

The return type has been correctly updated to Rawis | undefined to align with the database schema.


217-222: Updated return type for getNarratorInfo.

The function now properly returns Sources[] instead of the custom InfoSource[].


224-230: Updated return type for getSuccessors.

The function now returns Rawis[] which directly aligns with the database schema.


232-238: Updated return type for getPredecessors.

The function now returns Rawis[] which directly maps to the database schema.

@github-actions
Copy link

Cloudflare Pages Deployment

Environment: preview
Project: isnad
Built with commit: ffa3f28
Preview URL: https://83126f3d.isnad-acg.pages.dev
Branch Preview URL: https://sql-ts.isnad-acg.pages.dev

Wrangler Output

Uploading... (12405/17317)
Uploading... (12461/17317)
Uploading... (12501/17317)
Uploading... (12544/17317)
Uploading... (12712/17317)
Uploading... (13019/17317)
Uploading... (13489/17317)
Uploading... (14092/17317)
Uploading... (14811/17317)
Uploading... (15950/17317)
Uploading... (17317/17317)
✨ Success! Uploaded 4912 files (12405 already uploaded) (27.28 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://83126f3d.isnad-acg.pages.dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document nullable fields in the DB schema and auto generate typescript schema from it

2 participants