arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.16282cs.CL
收藏

通过超网络实现即时个性化大型语言模型自适应

Instant Personalized Large Language Model Adaptation via Hypernetwork

University of Notre Dame(诺丁汉大学);Amazon.com Inc(亚马逊公司);Université de Montréal(蒙特利尔大学)

Zhaoxuan Tan, Zixuan Zhang, Haoyang Wen, Zheng Li, Rongzhi Zhang, Pei Chen, Fengran Mo, Zheyuan Liu, Qingkai Zeng, Qingyu Yin, Meng Jiang

AI总结:

提出Profile-to-PEFT框架,使用超网络将用户编码直接映射到适配器参数,实现无需用户训练的即时个性化,在降低计算成本的同时优于现有方法。

摘要:

个性化大型语言模型(LLM)利用用户档案或历史记录来定制符合个人偏好的内容。然而,现有的参数高效微调(PEFT)方法,例如“每用户一个PEFT”(OPPU)范式,需要为每个用户训练单独的适配器,这使得它们在计算上昂贵且不适用于实时更新。我们引入了Profile-to-PEFT,一个可扩展的框架,它采用端到端训练的超网络,将用户编码档案直接映射到一组完整的适配器参数(例如LoRA),从而消除了部署时的每用户训练。这种设计实现了即时自适应、对未见用户的泛化以及保护隐私的本地部署。实验结果表明,我们的方法在部署时使用显著更少的计算资源,同时优于基于提示的个性化和OPPU。该框架对分布外用户表现出强大的泛化能力,并在不同的用户活动水平和不同的嵌入骨干下保持鲁棒性。所提出的Profile-to-PEFT框架实现了高效、可扩展且自适应的LLM个性化,适用于大规模应用。

英文摘要:

Personalized large language models (LLMs) tailor content to individual preferences using user profiles or histories. However, existing parameter-efficient fine-tuning (PEFT) methods, such as the ``One-PEFT-Per-User'' (OPPU) paradigm, require training a separate adapter for each user, making them computationally expensive and impractical for real-time updates. We introduce Profile-to-PEFT, a scalable framework that employs a hypernetwork, trained end-to-end, to map a user's encoded profile directly to a full set of adapter parameters (e.g., LoRA), eliminating per-user training at deployment. This design enables instant adaptation, generalization to unseen users, and privacy-preserving local deployment. Experimental results demonstrate that our method outperforms both prompt-based personalization and OPPU while using substantially fewer computational resources at deployment. The framework exhibits strong generalization to out-of-distribution users and maintains robustness across varying user activity levels and different embedding backbones. The proposed Profile-to-PEFT framework enables efficient, scalable, and adaptive LLM personalization suitable for large-scale applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑