大语言模型的个性化对齐
Personality Alignment of Large Language Models
AI总结：
本文提出个性化对齐概念，并构建含超32万被试的PAPI数据集及PAS激活干预优化方法，使大语言模型以更低成本高效匹配个体用户偏好，推动以人为中心的AI发展。
摘要：
大语言模型（LLMs）的对齐通常旨在反映一般性的人类价值观和行为，但往往无法捕捉个体用户独特的特征与偏好。为弥补这一不足，我们提出了个性化对齐（Personality Alignment）的概念。该方法针对个体用户或密切相关群体的特定偏好来定制大语言模型的响应与决策。受心理测量学启发，我们构建了基于人格量表的个性化对齐数据集PAPI，该数据集包含来自超过320,000名真实被试的多项人格评估数据，涵盖大五人格因素和黑暗三联征特质。这一全面的数据集使得能够在大语言模型上对积极和潜在问题人格维度两方面的对齐能力进行定量评估。考虑到个性化对齐所面临的挑战，如个人数据有限、偏好多样以及可扩展性需求，我们开发了一种激活干预优化方法。该方法以最少的数据和计算资源，增强了大语言模型高效对齐个体行为偏好的能力。值得注意的是，我们的方法PAS在取得优越性能的同时，所需的优化时间仅为DPO的1/5，为个性化对齐提供了实用价值。我们的工作为未来人工智能系统以真正个性化的方式进行决策和推理铺平了道路，增强了人工智能交互对每位用户的相关性和意义，并推动了以人为中心的人工智能发展。数据集和代码已在https://github.com/zhu-minjun/PAlign发布。
英文摘要：
Aligning large language models (LLMs) typically aim to reflect general human values and behaviors, but they often fail to capture the unique characteristics and preferences of individual users. To address this gap, we introduce the concept of Personality Alignment. This approach tailors LLMs' responses and decisions to match the specific preferences of individual users or closely related groups. Inspired by psychometrics, we created the Personality Alignment with Personality Inventories (PAPI) dataset, which includes data from over 320,000 real subjects across multiple personality assessments, including both the Big Five Personality Factors and Dark Triad traits. This comprehensive dataset enables quantitative evaluation of LLMs' alignment capabilities across both positive and potentially problematic personality dimensions. Recognizing the challenges of personality alignments, such as limited personal data, diverse preferences, and scalability requirements, we developed an activation intervention optimization method. This method enhances LLMs' ability to efficiently align with individual behavioral preferences using minimal data and computational resources. Remarkably, our method, PAS, achieves superior performance while requiring only 1/5 of the optimization time compared to DPO, offering practical value for personality alignment. Our work paves the way for future AI systems to make decisions and reason in truly personality ways, enhancing the relevance and meaning of AI interactions for each user and advancing human-centered artificial intelligence. The dataset and code are released at https://github.com/zhu-minjun/PAlign.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
