Skip to content

more accurate but slower#1

Open
WafiWadud wants to merge 2 commits intomattbatwings:mainfrom
WafiWadud:main
Open

more accurate but slower#1
WafiWadud wants to merge 2 commits intomattbatwings:mainfrom
WafiWadud:main

Conversation

@WafiWadud
Copy link

just some small adjustments

@OmerZeyveli
Copy link

Hey, noticed a couple of issues in this PR:

  • The to_fixed() scaling factor (10000000) seems excessively large and may cause overflow or loss of precision.

  • Using 10000000 as the bit-width in check_overflow() effectively disables proper overflow detection.

These changes might actually reduce accuracy instead of improving it. Maybe worth revisiting the fixed-point scaling approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments