arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.01604cs.AI
收藏

在真实环境中评估代理AI:故障模式、漂移模式及生产评估框架

Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework

Independent Researcher(独立研究者)

Mukund Pandey

AI总结:

本文针对代理AI在生产环境中持续运行时的评估挑战,提出七种故障模式分类及PAEF框架,揭示传统指标在检测故障模式上的不足。

摘要:

现有的大型语言模型评估框架,如HELM、MT-Bench、AgentBench和BIG-bench,都是为受控、单次会话、实验室规模的环境设计的。它们无法解决代理AI系统在生产环境中持续运行时出现的评估挑战:决策错误的累积、工具故障级联、非确定性输出漂移以及长期任务缺乏真实标注。本文做出了三个贡献。首先,我们提出了七种独特的生产代理系统故障模式分类,每种都基于在十亿事件规模系统上的观察。其次,我们通过实证证明标准指标——ROUGE、BERTScore、准确性/AUC以及上述代理基准——在检测每种故障模式时均失效。第三,我们提出了PAEF(生产代理评估框架),一个五维评估框架,具有开源参考实现,旨在对生产流量进行持续评估,而非单次基准运行。我们的分析显示,标准指标完全无法检测四种故障模式,而对其他三种故障模式的检测需要多个评估周期后才能显现。

英文摘要:

Existing evaluation frameworks for large language models -- including HELM, MT-Bench, AgentBench, and BIG-bench -- are designed for controlled, single-session, lab-scale settings. They do not address the evaluation challenges that emerge when agentic AI systems operate continuously in production: compounding decision errors, tool failure cascades, non-deterministic output drift, and the absence of ground truth for long-horizon tasks. This paper makes three contributions. First, we present a taxonomy of seven failure modes unique to production agentic systems, each grounded in observations from systems operating at billion-event scale. Second, we demonstrate empirically where standard metrics -- ROUGE, BERTScore, accuracy/AUC, and the agentic benchmarks above -- fail to detect each failure mode. Third, we propose PAEF (Production Agentic Evaluation Framework), a five-dimension evaluation framework with an open-source reference implementation, designed for continuous evaluation on production traffic rather than episodic benchmark runs. Our analysis shows that standard metrics fail to detect four of the seven failure modes entirely and detect three others only after a lag of multiple evaluation cycles.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑