arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.07689cs.MA
收藏

智能体交付工程预测可靠性框架

Agent Delivery Engineering Predictive Reliability Framework

Shanghai Qijing Digital Technology Co., Ltd.(上海启静数字科技有限公司)

Dexing Liu

AI总结:

针对长期语言模型多智能体系统的可靠性风险,提出ADE预测可靠性框架,聚合异构信号为信任边际指标,用三重方法并行预测,经多轮验证和实验,能检测“虚假繁荣”,实现可靠性量化并提供前瞻性警告。

摘要:

长期的语言模型多智能体系统面临基础设施监测无法察觉的可靠性风险。我们提出了ADE预测可靠性框架(ADE - PRF),它能从被动的退化检测中实现主动的健康轨迹预测。ADE - PRF将五层中的20个异构信号聚合为一个信任边际(TM)指标(39.2点动态范围)。三重方法并行预测可实现8小时预测:指数方法的平均绝对误差(MAE)=1.228,方向准确率=76.8%,99.65%在±10点公差范围内。生产验证跨越15个连续日的六个智能体配置文件中的380,227次预测和280,579次验证,以及七个沙盒控制实验。关键发现包括检测到“虚假繁荣”,以及在集成ADE插件后TM与真实状态的即时耦合,20个因素中有16个依赖于ADE收集的数据。指数方法始终优于卡尔曼方法。ADE - PRF为生产语言模型智能体提供了最早的可靠性量化和前瞻性警告。

英文摘要:

Long-horizon LLM multi-agent systems face reliability risks invisible to infrastructure monitoring. We propose the ADE Predictive Reliability Framework (ADE-PRF), enabling proactive health trajectory prediction from passive degradation detection. ADE-PRF aggregates 20 heterogeneous signals across five layers into a Trust Margin (TM) metric (39.2-point dynamic range). Triple-method parallel prediction enables 8-hour forecasts: the Exponential method achieves MAE=1.228, Direction Accuracy=76.8%, with 99.65% within +/-10-point tolerance. Production validation spans 380,227 predictions and 280,579 validations across six agent profiles over 15 continuous days, plus seven sandbox-controlled experiments. Key findings include detection of "false prosperity" -- degradation concealed by normal surface metrics -- and immediate TM coupling with ground-truth states upon ADE plugin integration, with 16/20 factors relying on ADE-collected data. Exponential consistently outperforms Kalman. ADE-PRF provides among the earliest reliability quantification with forward-looking warnings for production LLM agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑