-
Notifications
You must be signed in to change notification settings - Fork 7
Open
Labels
Benchmark-ideaNew benchmark type, new runner or new topic on existing runner or new scorereNew benchmark type, new runner or new topic on existing runner or new scorereenhancementNew feature or requestNew feature or request
Description
While translating some larger documents I noticed have much faster some models are. It would be pretty easy to ask models to translate large files we have generated where we actually actually only have validation data on a few specific sentences with validated correct translation to the other language we validate that those have correct translation but mostly we are look to benchmark the speed of translation of large documents. We should ensure that the input documents have nice paragraph numbering so we can easily find our validation sentences via the numbering.
coderabbitai
Metadata
Metadata
Assignees
Labels
Benchmark-ideaNew benchmark type, new runner or new topic on existing runner or new scorereNew benchmark type, new runner or new topic on existing runner or new scorereenhancementNew feature or requestNew feature or request