Skip to content

Migrating hfopenllmv2, global-mmlu-lite, livecodebenchpro to v0.2.0 #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nelaturuharsha merged 6 commits into from
Feb 11, 2026
Merged

Migrating hfopenllmv2, global-mmlu-lite, livecodebenchpro to v0.2.0 #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
af911c4
bump global-mmlu-lite and hfopenllm_v2 to v0.2.0
nelaturuharsha Feb 10, 2026
b871c5f
Merge branch 'main' of https://github.com/evaleval/every_eval_ever in…
nelaturuharsha Feb 10, 2026
9e6d461
delete duplicates in livecodebenchpro, latest schema changes
nelaturuharsha Feb 10, 2026
758070b
Merge branch 'main' of https://github.com/evaleval/every_eval_ever in…
nelaturuharsha Feb 11, 2026
e0ca88a
update global-mmlu-lite, make confidence_level optional (for now) and…
nelaturuharsha Feb 11, 2026
f36d435
chore: update source_metadata
nelaturuharsha Feb 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
8 changes: 3 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,10 +188,8 @@ Each evaluation (e.g., `livecodebenchpro`, `hfopenllm_v2`) has its own directory

Run following bash commands to generate pydantic classes for `eval.schema.json` and `instance_level_eval.schema.json` (to easier use in data converter scripts):

uv run datamodel-codegen --input eval.schema.json --output eval_types.py --class-name EvaluationLog --output-model-type pydantic_v2.BaseModel --input-file-type jsonschema --formatters ruff-format ruff-check
uv run datamodel-codegen --input instance_level_eval.schema.json --output instance_level_types.py --class-name InstanceLevelEvaluationLog --output-model-type pydantic_v2.BaseModel --input-file-type jsonschema --formatters ruff-format ruff-check

```bash
uv run datamodel-codegen --input eval.schema.json --output eval_types.py --class-name EvaluationLog --output-model-type pydantic_v2.BaseModel --input-file-type jsonschema
```
and
```bash
uv run datamodel-codegen --input instance_level_eval.schema.json --output instance_level_types.py --class-name InstanceLevelEvaluationLog --output-model-type pydantic_v2.BaseModel --input-file-type jsonschema
```
Loading
Loading