LoFA: Learning to Predict Personalized Priors for Fast Adaptation of Visual Generative Models

_{Official PyTorch Implementation}

---

Yiming Hao^1*, Mutian Xu^1*, Chongjie Ye^2,3,1, Jie Qin², Shunlin Lu², Yipeng Qin², Xiaoguang Han^1,2,3†
*equal contribution; †project lead
¹SSE, CUHKSZ  ²FNii-Shenzhen

³Guangdong Provincial Key Laboratory of Future Networks of Intelligence, CUHKSZ

⁴SDS, CUHKSZ ⁵Cardiff University

---

🔥 News

• [2025.12.19] 📄✨ The paper is officially released,training and inference pipelines will be released soon this month.

🧠 Overview

We introduce LoFA, a general framework that predicts personalized priors (i.e., LoRA weights) within seconds for fast adaptation of visual generative models and achieves performance comparable to, and even exceeding, conventional LoRA training.

TODO

• 🧪 Release inference pipeline
• 📦 Provide pretrained LoFA checkpoints
  • Text Conditioned Human Action Video Generation
  • Pose Conditioned Human Action Video Generation
  • Text-to-Video Stylization
  • Identity-Personalized Image Generation
• 🚀 Release Training pipeline
• 🧩 Add custom dataset support
• 📊 Release evaluation scripts

📄 Citation

If you use this work in your research, please cite our paper:

@misc{hao2025lofalearningpredictpersonalized,
      title={LoFA: Learning to Predict Personalized Priors for Fast Adaptation of Visual Generative Models}, 
      author={Yiming Hao and Mutian Xu and Chongjie Ye and Jie Qin and Shunlin Lu and Yipeng Qin and Xiaoguang Han},
      year={2025},
      eprint={2512.08785},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.08785}
}

---

📧 Contact

For questions and issues, please open an issue on GitHub or contact the authors.

---

_{Made with ❤️ by the GAP LAB}