ServiceNow

AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling

AgentJudgeBench：用于评估智能体工具调用的多难度基准

Abhigya Verma, Amit Kumar Saha, Seganrasan Subramanian, Sai Harshitha Aluru

机构 * ServiceNow AI（ServiceNow人工智能部门） 纠错

AI总结 该研究提出AgentJudgeBench基准，评估LLM评判在智能体工具调用工作流上的可靠性，发现其对齐度随难度下降，真实值并非总有益，还给出了实用评估指南。

Comments 31 Pages, 9 Figures, 31 Tables, Accepted as a main conference paper at EMNLP 2026