通过对比个人偏好实现个性化大语言模型解码
Personalized LLM Decoding via Contrasting Personal Preference
AI总结：
本文提出CoPe方法，通过奖励引导解码实现个性化，提升ROUGE-L指标10.57%。
摘要：
随着大型语言模型（LLMs）在各种现实世界应用中的逐步部署，LLMs的个性化变得越来越重要。尽管各种LLM个性化方法如基于提示的方法和基于训练的方法已被积极探讨，但有效解码时间算法的发展仍然被忽视，尽管它们已显示出潜力。在本文中，我们提出了CoPe（对比个人偏好），一种应用于在用户特定数据上进行参数高效微调（PEFT）后的解码时间方法。我们的核心思想是通过最大化每个用户的隐含奖励信号，利用奖励引导的解码来实现个性化。我们评估了CoPe在五个开放的个性化文本生成任务上的表现。我们的实验证果表明，CoPe实现了强大的性能，在ROUGE-L上平均提高了10.57%的个性化程度，无需依赖外部奖励模型或额外的训练过程。
英文摘要：
As large language models (LLMs) are progressively deployed in various real-world applications, personalization of LLMs has become increasingly important. While various approaches to LLM personalization such as prompt-based and training-based methods have been actively explored, the development of effective decoding-time algorithms remains largely overlooked, despite their demonstrated potential. In this paper, we propose CoPe (Contrasting Personal Preference), a novel decoding-time approach applied after performing parameter-efficient fine-tuning (PEFT) on user-specific data. Our core idea is to leverage reward-guided decoding specifically for personalization by maximizing each user's implicit reward signal. We evaluate CoPe across five open-ended personalized text generation tasks. Our empirical results demonstrate that CoPe achieves strong performance, improving personalization by an average of 10.57% in ROUGE-L, without relying on external reward models or additional training procedures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
