arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.26152cs.SE
收藏

为大语言模型系统设计的AI可观测性:从置信度校准到基础设施追踪的多层分析

AI Observability for Large Language Model Systems: A Multi-Layer Analysis of Monitoring Approaches from Confidence Calibration to Infrastructure Tracing

Red Hat(红帽)

Twinkll Sisodia

AI总结:

本文分析了大语言模型在生产环境中的可观测性需求,探讨了从置信度校准到基础设施追踪的多层监控方法,识别了当前领域中的关键缺口。

摘要:

大语言模型(LLMs)在生产环境中的部署催生了需要覆盖整个堆栈的可观测性系统,从模型内部到GPU内核。然而,现有监控方法仅关注堆栈的孤立层,缺乏对这些技术之间关系、重叠和互补的全面分析。本文对五项最近的研究贡献(2025-2026)进行了结构化分析,这些贡献共同定义了新兴的AI可观测性领域:通过强化学习进行置信度校准(MIT)、通过命题探测进行内部状态监控(UC Berkeley)、链式思维可监控性评估(OpenAI)、自主云操作基准测试(Microsoft Research, UC Berkeley, UIUC)以及非侵入式推理级追踪(TRUFFLD)。我们将这些贡献组织成五层可观测性分类法,综合其关键发现形成统一比较,并识别出四个尚未解决的关键缺口。我们进一步将这些研究方向与实际的操作可观测性系统相结合,将基础设施遥测转化为运维团队可操作的见解。我们的分析表明,尽管单个监控层发展迅速,但将模型级置信信号与基础设施级异常连接起来,形成连贯的操作智能,仍然是该领域的主要开放问题。

英文摘要:

The deployment of large language models (LLMs) in production environments has created an urgent need for observability systems that span the full stack -- from model internals to GPU kernels. Yet existing monitoring approaches address isolated layers of this stack, and no comprehensive analysis has examined how these techniques relate, overlap, or complement each other. This paper presents a structured analysis of five recent research contributions (2025-2026) that collectively define the emerging landscape of AI observability: confidence calibration via reinforcement learning (MIT), internal state monitoring through propositional probes (UC Berkeley), chain-of-thought monitorability evaluation (OpenAI), autonomous cloud operations benchmarking (Microsoft Research, UC Berkeley, UIUC), and non-intrusive inference-level tracing (TRUFFLD). We organize these contributions into a five-layer observability taxonomy, synthesize their key findings into a unified comparison, and identify four critical gaps that remain unaddressed. We further contextualize these research directions against practical operational observability systems that translate infrastructure telemetry into actionable insights for site reliability teams. Our analysis reveals that while individual monitoring layers have matured rapidly, the integration challenge -- connecting model-level confidence signals with infrastructure-level anomalies into coherent operational intelligence -- remains the defining open problem for the field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑