AI 中文总结
该研究针对多阶段LLM流水线提出证据状态可靠性评估框架，发现受控退化下结构符合性可改善但证据敏感的阶段成功会恶化，且退化证据的检测与恢复存在偏差。
AI 中文摘要
多阶段大语言模型（LLM）流水线即便下游阶段可用证据出现不完整、压缩或冲突，仍可保持结构有效。本文引入并实施了证据状态可靠性（Evidence-State Reliability, ESR），这是一种评估层，关注中间证据是否足够完整、有依据、内部一致且可用于对应阶段的指定功能。ESR与衡量结构符合性的解析器有效性分开评估。我们使用GLM-5.2在60个经过清理的基础案例上，针对四种证据条件（干净、有损压缩、部分丢弃、含噪冲突）评估该框架，每种条件均经过决策、审计和升级三个阶段处理。该设计包含720个计划且已记录的调用，最终保留并清理了713条执行行。在9组匹配的退化-干净条件-阶段对比中，所有实施的阶段成功估计值均为负，且所有95%自助法置信区间均保持在零以下；所有9个解析器有效性点估计值均为正，不过3个部分丢弃条件的置信区间包含零。在解析器有效的退化审计输出中，各退化条件下的退化检测率均为1.0，但虚假保证率不为零；在解析器有效的退化升级输出中，各退化条件下的恢复率均为0.0。结果显示，所评估流水线中存在有界的可靠性层偏差：在相同受控干预下，结构符合性可呈方向改善，而对证据敏感的阶段成功则会恶化，同时还可区分退化证据的检测与恢复。结论仅适用于所评估的模型配置、流水线设计、选定的清理案例、评分程序及单次规模化运行。
英文摘要
Multi-stage LLM pipelines can remain structurally valid even when evidence available to downstream stages becomes incomplete, compressed, or conflicting. This paper introduces and operationalizes Evidence-State Reliability (ESR), an evaluation layer concerned with whether intermediate evidence remains sufficiently complete, grounded, internally consistent, and usable for a stage's assigned function. ESR is evaluated separately from parser validity, which measures structural conformance. We evaluate the framework using GLM-5.2 on 60 sanitized base cases under four evidence conditions: clean, compressed-lossy, partial-dropout, and noisy-conflicting. Each condition was processed through decision, audit, and escalation stages. The design comprised 720 planned and ledgered calls, with 713 retained, sanitized execution rows. Across nine matched degraded-minus-clean condition-stage comparisons, all operational stage-success estimates were negative, and all 95% bootstrap intervals remained below zero. All nine parser-validity point estimates were positive, although the three partial-dropout intervals included zero. Among parser-valid degraded audit outputs, degradation detection was 1.0 in each degraded condition, while false-assurance rates remained non-zero; among parser-valid degraded escalation outputs, recovery was 0.0 in every degraded condition. The results show a bounded reliability-layer divergence in the evaluated pipeline: structural conformance can improve directionally while evidence-sensitive stage success deteriorates under the same controlled intervention. They also separate detection of degraded evidence from recovery. The conclusions are limited to the evaluated model configuration, pipeline design, selected sanitized cases, scoring procedure, and single scaled run.
Comments32 pages, 4 figures, 6 tables. Code and reproducibility materials: https://github.com/NaimurRahmanR/evidence-state-reliability
