重新思考在词级别上的大语言模型个性化
Rethinking Personalization in Large Language Models at the Token Level
AI总结：
PerContrast通过自对比方法和PerCE损失提升大语言模型在词级别上的个性化性能，实现平均收益超10%并具有良好的迁移能力。
摘要：
随着大语言模型（LLMs）现在在多样化任务上表现出色，人们对其输出进行个性化以满足个别用户需求的需求日益增长。个性化通常被框架为在基础NLP任务之上的额外一层，要求模型响应在满足用户特定需求的同时完成底层任务。从词级别的角度来看，响应中的不同词对个性化贡献的程度各不相同。因此，那些具有更高个性化相关性的词在开发个性化LLM时应受到更大的重视。然而，准确估计此类个性化程度仍具有挑战性。为了解决这一挑战，我们提出了PerContrast，一种自对比方法，通过因果干预来估计每个输出词对用户特定信息的依赖性。在此机制的基础上，我们开发了PerCE损失，通过bootstrap过程，使模型在训练过程中自适应地提高估计个性化程度较高的词的权重，使模型能够在估计和优化这些词之间交替进行。在多个LLM上的实验表明，PerCE在个性化性能上取得了显著提升，且额外成本极低，平均收益超过10%，在LongLaMP数据集上最高达到68.04%，同时具有良好的跨任务和跨场景迁移能力。这些结果突显了词级别个性化建模的重要性，并确立了词感知训练作为推进个性化LLM的有效范式。
英文摘要：
With large language models (LLMs) now performing strongly across diverse tasks, there is growing demand for them to personalize outputs for individual users. Personalization is typically framed as an additional layer on top of a base NLP task, requiring model responses to meet user-specific needs while still accomplishing the underlying task. From a token-level perspective, different tokens in a response contribute to personalization to varying degrees. Tokens with higher personalization relevance should therefore receive greater emphasis when developing personalized LLMs. However, accurately estimating such personalization degrees remains challenging. To address this challenge, we propose PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention. Building on this mechanism, we develop the PerCE loss, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens. Experiments on multiple LLMs demonstrate that PerCE substantially improves personalization performance with minimal additional cost, achieving average gains of over 10% and up to 68.04% on the LongLaMP dataset, along with strong cross-task and cross-scenario transferability. These results highlight the importance of token-level personalization modeling and establish token-aware training as a simple yet effective paradigm for advancing personalized LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
