复杂约束与超越：RLVR的专家评分标准
ComplexConstraints and Beyond: Expert Rubrics for RLVR
AI总结：
提出专家设计的评分标准作为评估和训练信号，通过复杂指令遵循和企业智能体任务验证，在RL训练中显著提升模型性能。
摘要：
随着LLM能力的快速提升，用于评估它们的方法越来越滞后。传统基准依赖于对狭窄、表面约束的程序化验证，但现实世界的指令遵循和智能体任务需要评估细微的、上下文依赖的行为，这些行为难以通过简单的脚本检查。我们提出了一个基于专家策划的评分标准评估的系统分析作为替代范式，借鉴了来自两个领域的实证证据：复杂指令遵循和企业智能体任务。我们首先阐述了构建高质量评分标准的五个设计原则，包括最大可行原子性、意图感知标准设计和迭代LLM判断校准。为了验证这些原则，我们引入了ComplexConstraints，一个新的专家策划的指令遵循数据集，其中每个提示与10-40个原子评分标准配对。我们证明这些专家评分标准不仅是更好的评估工具，而且是高度有效的训练信号：在大约1000个ComplexConstraints示例上训练，使得4B参数模型在指令遵循上提升+15.5%，235B参数模型提升+12.2%，而在评分标准评分的企业环境上进行单周期RL训练产生的收益可以转移到模型从未训练过的分布外基准（BFCL +4.5%，Tau2-Bench +7.4%，Tool-Decathlon +6.8%）。我们的发现表明，专家编写的评分标准既改进了前沿LLM能力的测量，也改进了其发展，作为有效的评估和RL训练信号。
英文摘要：
Evaluation protocols can lag behind LLM capabilities. Programmatically verified benchmarks cover narrow surface constraints, whereas real-world instruction following and agentic workflows require judging semantic, contextual, and policy-dependent behavior. We study expert-curated rubric-based evaluation as a unified mechanism for measurement and reinforcement-learning rewards across two settings: complex instruction following and enterprise agentic tasks. We identify rubric-design choices that affect reward quality, including maximum viable atomicity, intent-aware criterion design, and LLM-judge calibration. We introduce ComplexConstraints, an expert-curated instruction-following suite comprising a public 75-prompt benchmark with 1,559 rubric criteria and a disjoint 1,000-prompt training set, with 10-40 atomic criteria per prompt. Empirically, rubric rewards improve training in both fixed task datasets, such as ComplexConstraints, and stateful RL environments, such as CoreCraft. Training a 4B model on ComplexConstraints improves mean criterion pass rate by +15.5 pp on a held-out split, bringing it within 0.5 pp of the untrained baseline of a roughly 60x larger Qwen3 model, and the gains transfer to external benchmarks the model never saw during training: +8.4 pp on AdvancedIF and +10.1 pp on MultiChallenge. In CoreCraft, rubric-reward RL likewise transfers to out-of-distribution benchmarks (+4.5 pp BFCL, +7.4 pp tau^2-Bench, +6.8 pp Toolathlon). These results show that expert-authored rubrics provide effective evaluation targets and scalable reward signals for improving LLM instruction following and agentic behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
