experimental scores

[WEYAI]

Could you please help me answer this? I'm currently trying to replicate the experiments in the paper.
I would like to inquire about the experimental scores in the paper. For instance, for the game "Sokoban", was it run using the "harness: true" ， "observation_mode: vision" settings ?

[snyhlxde1]

Hi there. thanks for your interest in our work.

If you want to reproduce the row without harness, please set harness: false in the config. Conversely you can set harness: true. And yes both with observation mode: vision to add rendered img as part of environment observation.

[WEYAI]

In the leaderboard on Hugging Face, are all games reproduced with the parameters harness: true, observation_model: vison? I've noticed that the default configuration in our code isn't set this way; some settings use observation_model: text, and I've found that text scores higher. So, how should I set the parameters to reproduce the leaderboard? Thank you for your hard work; this is an excellent piece of work.