Filter Threshold in scripts/internal_knowledge_preparation.py

[HarshSaini10]

According to the research paper, the following line : "We directly adopted the evaluator again for knowledge strips filtering, and the top-k is set to 5, filter threshold as -0.5", mentions that a threshold of -0.5 was used to filter the strips, but it is nowhere to be found in the internal_knowledge_preparation.py script.

[HuskyInSalt]

We chose the top-k filtering strategy in the repo. You can change the strategy by modifying the function select_relevants and setting the threshold. We will also update the second strategy later.