LatentAudit: 用于检索增强生成的实时白盒可信度监控与可验证部署
LatentAudit: Real-Time White-Box Faithfulness Monitoring for Retrieval-Augmented Generation with Verifiable Deployment
AI总结：
LatentAudit通过分析生成器中间层激活来实时监控RAG系统的可信度，无需额外模型，且在多个基准测试中表现出高准确率和低延迟，支持公开验证。
摘要：
检索增强生成（RAG）减轻了幻觉问题，但无法完全消除：部署系统仍需在推理时决定其答案是否由检索到的证据支持。我们引入了LatentAudit，一种白盒审计器，通过池化中到晚期残差流激活，并测量其与证据表示的马氏距离。所得二次规则无需辅助判断模型，在生成时运行，且足以在小数据集上校准。我们展示了残差流几何携带可利用的可信度信号，该信号在架构变化和现实检索失败中保持稳定，且相同规则仍可进行公开验证。在PubMedQA上使用Llama-3-8B时，LatentAudit达到0.942 AUROC，延迟为0.77毫秒。在三个问答基准和五个模型家族（Llama-2/3，Qwen-2.5/3，Mistral）上，监控器保持稳定；在四维压力测试中，面对矛盾、检索遗漏和部分支持噪声，其在PubMedQA上达到0.9566-0.9815 AUROC，在HotpotQA上达到0.9142-0.9315。在16位固定定点精度下，审计规则保留99.8%的FP16 AUROC，使基于Groth16的公开验证成为可能，无需暴露模型权重或激活。这些结果将残差流几何确立为实时RAG可信度监控和可验证部署的实用基础。
英文摘要：
Retrieval-augmented generation (RAG) mitigates hallucination but does not eliminate it: a deployed system must still decide, at inference time, whether its answer is actually supported by the retrieved evidence. We introduce LatentAudit, a white-box auditor that pools mid-to-late residual-stream activations from an open-weight generator and measures their Mahalanobis distance to the evidence representation. The resulting quadratic rule requires no auxiliary judge model, runs at generation time, and is simple enough to calibrate on a small held-out set. We show that residual-stream geometry carries a usable faithfulness signal, that this signal survives architecture changes and realistic retrieval failures, and that the same rule remains amenable to public verification. On PubMedQA with Llama-3-8B, LatentAudit reaches 0.942 AUROC with 0.77,ms overhead. Across three QA benchmarks and five model families (Llama-2/3, Qwen-2.5/3, Mistral), the monitor remains stable; under a four-way stress test with contradictions, retrieval misses, and partial-support noise, it reaches 0.9566--0.9815 AUROC on PubMedQA and 0.9142--0.9315 on HotpotQA. At 16-bit fixed-point precision, the audit rule preserves 99.8% of the FP16 AUROC, enabling Groth16-based public verification without revealing model weights or activations. Together, these results position residual-stream geometry as a practical basis for real-time RAG faithfulness monitoring and optional verifiable deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
