群体偏好对齐：基于原位对话的定制化大语言模型响应生成
Group Preference Alignment: Customized LLM Response Generation from In-Situ Conversations
AI总结：
本文提出群体偏好对齐框架GPA，从真实对话中提取群体偏好准则，并通过情境调优推理和准则微调推理生成定制化响应，在保持标准基准性能的同时显著提升用户偏好对齐效果。
摘要：
大语言模型（LLMs）由于其“一刀切”的训练范式，往往无法满足不同用户群体的专门需求，且关于各群体期望何种个性化方面的研究十分有限。为应对这些局限，我们提出了一种群体感知的个性化框架——群体偏好对齐（Group Preference Alignment, GPA），该框架识别不同用户群体之间对话偏好的情境特定差异，并引导大语言模型去满足这些偏好。我们的方法包含两个步骤：（1）群体感知偏好提取：从真实世界对话日志中提取差异最大的用户群体偏好，并将其提炼为可解释的评分准则；（2）定制化响应生成：通过两种方法利用这些评分准则：a）情境调优推理（GPA-CT），通过依赖情境的提示指令动态调整响应；b）准则微调推理（GPA-FT），利用评分准则生成对比性合成数据，通过对齐实现群体特定模型的个性化。实验表明，我们的框架显著提升了输出与用户偏好之间的一致性，并优于基线方法，同时在标准基准上保持了稳健的性能。
英文摘要：
LLMs often fail to meet the specialized needs of distinct user groups due to their one-size-fits-all training paradigm \cite{lucy-etal-2024-one} and there is limited research on what personalization aspects each group expect. To address these limitations, we propose a group-aware personalization framework, Group Preference Alignment (GPA), that identifies context-specific variations in conversational preferences across user groups and then steers LLMs to address those preferences. Our approach consists of two steps: (1) Group-Aware Preference Extraction, where maximally divergent user-group preferences are extracted from real-world conversation logs and distilled into interpretable rubrics, and (2) Tailored Response Generation, which leverages these rubrics through two methods: a) Context-Tuned Inference (GAP-CT), that dynamically adjusts responses via context-dependent prompt instructions, and b) Rubric-Finetuning Inference (GPA-FT), which uses the rubrics to generate contrastive synthetic data for personalization of group-specific models via alignment. Experiments demonstrate that our framework significantly improves alignment of the output with respect to user preferences and outperforms baseline methods, while maintaining robust performance on standard benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
