是否每个用户都需要私有LoRA？将个性化与逐用户适配解耦
Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation
浏览论文内容
中文总结 AI 辅助
本研究提出LINEUP框架，通过共享低秩个性化因子库和极小的用户编码（每用户仅8个标量），在12项指标上全面超越私有LoRA，证明个性化可主要依赖可复用共享容量。
中文摘要 AI 辅助
个性化大语言模型通常需要为每个用户维护一个完整的适配状态。然而，随着用户群体的增长，这种范式扩展性较差。我们通过个性化容量分配的视角重新审视这一设计：有多少适配容量可以在用户间共享，共享容量应如何组合，以及必须保留多少用户特定容量。我们通过三项互补的实证分析来回答这些问题。我们发现，独立的用户适配器包含大量可跨用户复用的结构，可复用方向的有效性既反映了用户相关性也反映了查询间的变化，并且用户历史为紧凑的个体修正提供了可迁移的信号。受这些发现启发，我们提出了LINEUP。它学习一组可复用的低秩个性化因子库，通过用户条件召回和查询依赖校准进行组合，并将目标用户适配限制为共享修正空间上的一个极小的用户编码。该设计将表达性个性化容量与逐用户可训练状态解耦。每个目标用户仅优化八个标量，而所有共享组件保持固定。相比之下，所评估的私有LoRA配置使用每用户419万个参数。我们的理论分析给出了有限步、有限历史的风险界以及用户编码细化改善历史初始化的充分条件。在涵盖个性化分类、预测和生成的六项任务中，LINEUP在所有12项指标上均领先，每项指标在三次独立运行上取平均（例如，相对于最强基线，LaMP-3的RMSE降低了11.4%）。在历史有限的情况下，它仍保持优势。这些结果表明，丰富的个性化可以主要由可复用的、条件组合的共享容量来支持，而独立的用户适配则被限制在一个极小的修正状态中。
英文摘要
Personalized large language models often require a complete adaptation state for each user. However, this paradigm scales poorly as the user population grows. We revisit this design through the lens of personalization capacity allocation: how much adaptation capacity can be shared across users, how the shared capacity should be composed, and how much must remain user-specific. We answer them through three complementary empirical analyses. We find that independent user adapters contain substantial cross-user reusable structure, that the utility of reusable directions reflects both user relevance and variation across queries, and that user histories provide transferable signals for compact individual correction. Motivated by these findings, we propose LINEUP. It learns a bank of reusable low-rank personalization factors, composes them through user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny user code over a shared correction space. This design decouples expressive personalization capacity from per-user trainable state. Each target user optimizes only eight scalars, while all shared components remain fixed. By comparison, the evaluated private-LoRA configuration uses 4.19 million per-user parameters. Our theoretical analysis gives a finite-step, finite-history risk bound and sufficient conditions for user-code refinement to improve on history initialization. Across six tasks spanning personalized classification, prediction, and generation, LINEUP leads on all 12 metrics, each averaged over three independent runs (e.g., reducing LaMP-3 RMSE by 11.4% relative to the strongest baseline). It maintains advantages under limited history. These results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.
发表机构
- Samsung Semiconductor, US（三星半导体（美国））
- Rice University（莱斯大学）
机构由 AI 辅助整理，请以论文原文为准。
