P-Check：通过学习生成动态检查清单推进个性化奖励模型
P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic Checklist
AI总结：
本文提出P-Check框架，通过学习生成动态检查清单来提升个性化奖励模型的准确性，并引入偏好对比准则加权策略以增强个性化生成效果。
摘要：
近期个性化奖励建模方法主要依赖用户交互历史对齐模型判断与个体偏好，但现有方法往往将用户上下文视为静态或隐式条件信号，未能捕捉人类判断的动态性和多维性。本文提出P-Check，一种新颖的个性化奖励建模框架，旨在训练一个即插即用的检查清单生成器，合成动态评估标准以指导奖励预测。为更好地对齐这些清单与个性化细微差别，我们引入偏好对比准则加权，一种训练策略，根据其对个性化判断的判别能力分配显著性分数。我们进行了广泛实验，证明P-Check不仅提高了奖励准确性，还增强了下游个性化生成，并在OOD场景中保持鲁棒性。
英文摘要：
Recent approaches in personalized reward modeling have primarily focused on leveraging user interaction history to align model judgments with individual preferences. However, existing approaches largely treat user context as a static or implicit conditioning signal, failing to capture the dynamic and multi-faceted nature of human judgment. In this paper, we propose P-Check, a novel personalized reward modeling framework, designed to train a plug-and-play checklist generator that synthesizes dynamic evaluation criteria for guiding the reward prediction. To better align these checklists with personalized nuances, we introduce Preference-Contrastive Criterion Weighting, a training strategy that assigns saliency scores to criteria based on their discriminative power for personalized judgment. We conduct extensive experiments and demonstrate that P-Check not only improves reward accuracy but also enhances downstream personalized generation, and remains robust in OOD scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
