Sync forked dependencies with upstream

[waynexia]

This project currently contains many forked dependencies, make it hard to maintain and evolve. This ticket tracks how to replace them with upstream version or why we can't do that.

• flatbuffers
• vortex
• lance
• parquet-mine

[XinyuZeng]

Many of the forked dependencies are for paper experiments running. To let the project evolve, we'd better froze the current code state in a separate branch or tag (e.g., SIGMOD2026-reproduce), then delete all the paper-related code.

[waynexia]

That's great. Could you checkout a branch for this? (I don't have permission 🙈)

[XinyuZeng]

I will try to do a cleanup of all the dependency this week. I also think the file reader needs a refactor or even a rewrite to reach industry level strength (e.g., async API for datafusion). We can further discuss about the specific painpoints you mentioned to me when dealing with Parquet before and try solve them together during the rewrite.