Skip to content

Some data entries have mismatching #4

@fallcat

Description

@fallcat

Hi!

Thank you for the awesome work!

I'm attempting to use your benchmark and somehow found that many examples have mismatch in the number of sections being annotated and number of sections. Sometimes there are more annotations than number of sections and sometimes the opposite.

Image

Do you know what is the correct way to use this data when there is a mismatch?

Also, for some sections that have reasoning_correctness = '', does it mean it is having propagated error from previous steps or does it mean it is correct?

Thank you again for the great work!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions