arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18029cs.CLcs.AI
收藏

迈向更标准化的AI评估:从模型到代理

Towards More Standardized AI Evaluation: From Models to Agents

G42

Ali El Filali, Inès Bedar

AI总结:

本文探讨了AI评估从模型向代理转变的必要性,强调评估应作为测量学科而非表演舞台,以在非确定性系统中调节信任和治理。

摘要:

评估不再是机器学习生命周期中的最终检查点。随着AI系统从静态模型发展为复合、工具使用的代理,评估成为核心的控制功能。问题不再是“模型有多好?”而是“我们能否信任系统在变化、大规模情况下按预期行为?”然而,大多数评估实践仍根植于模型时代继承的假设:静态基准、聚合分数和一次性成功标准。本文认为,这些方法越来越模糊而非阐明系统行为。我们探讨评估流程本身如何引入无声的失败模式,为何高基准分数 routinely 欺骗团队,以及代理系统如何根本改变性能测量的意义。而不是提出新的指标或更难的基准,我们旨在明确评估在AI时代,尤其是对于代理的作用:不作为表演舞台,而作为一种测量学科,它在非确定性系统中调节信任、迭代和治理。

英文摘要:

Evaluation is no longer a final checkpoint in the machine learning lifecycle. As AI systems evolve from static models to compound, tool-using agents, evaluation becomes a core control function. The question is no longer "How good is the model?" but "Can we trust the system to behave as intended, under change, at scale?". Yet most evaluation practices remain anchored in assumptions inherited from the model-centric era: static benchmarks, aggregate scores, and one-off success criteria. This paper argues that such approaches are increasingly obscure rather than illuminating system behavior. We examine how evaluation pipelines themselves introduce silent failure modes, why high benchmark scores routinely mislead teams, and how agentic systems fundamentally alter the meaning of performance measurement. Rather than proposing new metrics or harder benchmarks, we aim to clarify the role of evaluation in the AI era, and especially for agents: not as performance theater, but as a measurement discipline that conditions trust, iteration, and governance in non-deterministic systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑