Skip to content

Conversation

@shubhamugare
Copy link
Collaborator

This issue resurfaced again with the last refactor.

Test:

python3 syncode/infer.py --mode grammar_strict --model google/gemma-2-2b-it --dataset json_eval --grammar json --max_new_tokens 400

The accuracy of JSON generation should be 99%. (This was reduced to 78% due to the bug)

  • Added a unit test to ensure that empty generation is diallowed

@shubhamugare shubhamugare merged commit 901e528 into main Nov 29, 2024
1 check passed
@shubhamugare shubhamugare deleted the fix8 branch December 27, 2024 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants