Skip to content

The MNLI score in lm-evaluation-harness #61

@wang99711123

Description

@wang99711123

Thanks for the great work!

I'm trying to reproduce the results you report. I downloaded the model weights from link https://huggingface.co/yahma/alpaca-7b-lora and evaluated them under the framework of lm-evaluation-harness. But I only got 41.7% accuracy on MNLI dataset.

When using lm-evaluation-harness, did you perform other data processing tricks to get 51.6% acc?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions