从100万用户到每一位用户：面向用户级对齐的可扩展个性化偏好
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment
AI总结：
针对现有LLM对齐忽视用户偏好多样性的问题，本文提出可扩展个性化对齐框架，含偏好空间与角色表征，并构建AlignX数据集及两种对齐方法，在四个基准上平均提升17.06%准确率。
摘要：
大型语言模型（LLMs）传统上通过一种一刀切的方法进行对齐，这种方法假设人类偏好是统一的，从根本上忽视了用户价值观和需求的多样性。本文介绍了一个用于LLMs可扩展个性化对齐的综合框架。我们建立了一个系统化的偏好空间，刻画心理和行为维度，同时引入多样化的角色表征，以在真实场景中进行稳健的偏好推断。在此基础之上，我们推出了\ extsc{AlignX}，一个包含超过130万个个性化偏好示例的大规模数据集，并开发了两种互补的对齐方法：\ extit{上下文内对齐}直接以角色表征为条件，以及\ extit{偏好桥接对齐}对中间偏好分布进行建模。大量实验表明，与现有方法相比，我们的方法有显著改进，在四个基准测试中平均准确率提升17.06%，同时展现出对新颖偏好的强大适应能力、对有限用户数据的稳健性以及精确的偏好可控性。这些结果验证了我们的方法在迈向用户自适应AI系统方面的有效性。
英文摘要：
Large language models (LLMs) have traditionally been aligned through one-size-fits-all approaches that assume uniform human preferences, fundamentally overlooking the diversity in user values and needs. This paper introduces a comprehensive framework for scalable personalized alignment of LLMs. We establish a systematic preference space characterizing psychological and behavioral dimensions, alongside diverse persona representations for robust preference inference in real-world scenarios. Building upon this foundation, we introduce \textsc{AlignX}, a large-scale dataset of over 1.3 million personalized preference examples, and develop two complementary alignment approaches: \textit{in-context alignment} directly conditioning on persona representations and \textit{preference-bridged alignment} modeling intermediate preference distributions. Extensive experiments demonstrate substantial improvements over existing methods, with an average 17.06\% accuracy gain across four benchmarks while exhibiting a strong adaptation capability to novel preferences, robustness to limited user data, and precise preference controllability. These results validate our approach toward user-adaptive AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
