当个性化遇上现实：个性化偏好学习的多面分析
When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning
AI总结：
本研究提出一个多面评估框架，衡量个性化偏好学习方法的性能、公平性、意外影响和适应性，实验发现用户分歧大时性能差异可达36%，且个性化可能引入20%安全错位，凸显整体评估的必要性。
摘要：
虽然基于人类反馈的强化学习（RLHF）被广泛用于使大型语言模型（LLMs）与人类偏好对齐，但它通常假设所有用户具有同质偏好，忽视了多样化的人类价值观和少数群体的观点。尽管个性化偏好学习通过为个体用户定制独立偏好来解决这一问题，但该领域缺乏标准化的方法来评估其有效性。我们提出了一个多方面的评估框架，不仅衡量性能，还衡量公平性、意外影响以及在不同偏好分歧程度下的适应性。通过在三组偏好数据集上对八种个性化方法进行广泛实验比较，我们证明当用户强烈分歧时，方法之间的性能差异可达36%，且个性化可能引入高达20%的安全错位。这些发现凸显了采用整体性评估方法以推动开发更有效、更具包容性的偏好学习系统的关键需求。
英文摘要：
While Reinforcement Learning from Human Feedback (RLHF) is widely used to align Large Language Models (LLMs) with human preferences, it typically assumes homogeneous preferences across users, overlooking diverse human values and minority viewpoints. Although personalized preference learning addresses this by tailoring separate preferences for individual users, the field lacks standardized methods to assess its effectiveness. We present a multi-faceted evaluation framework that measures not only performance but also fairness, unintended effects, and adaptability across varying levels of preference divergence. Through extensive experiments comparing eight personalization methods across three preference datasets, we demonstrate that performance differences between methods could reach 36% when users strongly disagree, and personalization can introduce up to 20% safety misalignment. These findings highlight the critical need for holistic evaluation approaches to advance the development of more effective and inclusive preference learning systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
