Improving the documentation for qaeval #141
Description
Hi,
thanks for sharing qaeval and your work on making metrics accessible! As I think making things accessible is very important to enhance further research and community dynamics I want to share my impression on setting up qaeval metric in my pipeline.
It seems the information is scattered around different places and it might be useful to be aligned and condensed.
- How to install: advantages / disadvantages and differences between qaeval-repro, sacre-rouge and repro are not entirely clear. (qa-eval suggests sacre rouge, but with docker or not? repro doesnt suggest sacre rouge, so use it directly?)
- Differences to paper: only on qaeval-repo
- Return specification: only documented in sacre rouge, but does not match the return type in repro (also containing "lerc")(so additional differences to paper?)