arXiv 2601.080972026-01cs.CLcs.LG

AdaJudge: 自适应多视角评判用于奖励建模

AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling

Emory University（埃默里大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

Yongliang Miao, Yangyang Liang, Mengnan Du

备注：: ACL 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出AdaJudge框架，通过门控精化块和自适应多视角池化模块，联合优化表示与聚合，解决奖励建模中静态归纳偏差和表示不匹配问题，在RM-Bench和JudgeBench上超越现有模型。

摘要：

奖励建模对于将大型语言模型与人类偏好对齐至关重要，但主流架构依赖静态池化策略将序列压缩为标量分数。然而，这种范式存在两个关键限制：静态归纳偏差与任务相关的偏好信号不匹配，以及表示不匹配，因为骨干网络针对生成的优化使其表示不适用于细粒度判别。为解决这一问题，我们提出AdaJudge，一个统一框架，联合调整表示和聚合。AdaJudge首先通过门控精化块将骨干网络表示改进到判别导向的空间。然后，它用自适应多视角池化模块替换静态读出，该模块动态路由并组合证据。在RM-Bench和JudgeBench上的大量实验表明，AdaJudge优于强大的现成奖励模型和传统池化基线。

英文摘要：

Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into scalar scores. This paradigm, however, suffers from two key limitations: a static inductive bias that misaligns with task-dependent preference signals, and a representational mismatch, as the backbone's optimization for generation leaves its representations ill-suited to fine-grained discrimination. To address this, we propose AdaJudge, a unified framework that jointly adapts representation and aggregation. AdaJudge first improves backbone representations into a discrimination-oriented space via gated refinement blocks. It then replaces the static readout with an adaptive multi-view pooling module, which dynamically routes and combines evidence. Extensive experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文