Files of seq2seq_construction

[YaooXu]

Could you provide the files of seq2seq_construction like USKG? Because, I found that there are some differences between the dataset constructed using the construction file provided by USKG and the one used in StructLM.

Thanks a lot!

[azhx]

Hi, I'm sorry that the code isn't cleaned and there is no easy way to run it at the moment, but please try and take a look here. https://github.com/azhx/uskg_eval/blob/main/build_llama_dataset.py

I believe the main difference from uskg is the fact that we use a different type of truncation scheme.
take a look at process_batch