SIRIN：用于检测检索增强与记忆基大型语言模型系统中上下文幻觉的统一工具包

中文摘要 AI 辅助

SIRIN（语义不一致识别与检查枢纽）是一款统一工具包及交互式网页用户界面，用于检测检索增强、智能体式与记忆基大型语言模型系统中的上下文幻觉（即由提供证据无法支撑的流畅合理响应）。SIRIN将三类检测器范式（表示探测、不确定性估计与判断式验证），以及生成前查询可回答性这一互补任务，整合至同一界面、配置系统与评估流水线中，支持白盒与黑盒设置下的响应级与片段级检查。该网页UI可通过幻觉分数、未支撑片段高亮及检测器并排对比，对用户提供的上下文-查询-答案三元组进行实时分析，采用轻量级插件设计以添加新检测器。我们在幻觉检测、查询可回答性及作为长期记忆系统内的忠实性门控场景中验证了SIRIN，其源代码公开于此https URL。

英文摘要

SIRIN (Semantic Inconsistency Recognition and Inspection Nexus) is a unified toolkit and interactive web UI for detecting contextual hallucinations (fluent, plausible responses unsupported by the provided evidence) in retrieval-augmented, agentic, and memory-grounded LLM systems. SIRIN unifies three detector paradigms (representation probing, uncertainty estimation, and judge-style verification) and the complementary task of pre-generation query answerability under one interface, configuration system, and evaluation pipeline, supporting response- and span-level inspection in both white-box and black-box settings. The web UI enables live analysis of user-supplied context-query-answer triples through hallucination scores, unsupported-span highlighting, and side-by-side detector comparison, with a lightweight plug-in design for adding new detectors. We demonstrate SIRIN on hallucination detection, query answerability, and as a faithfulness gate within long-term memory systems. The source code is publicly available at https://github.com/sb-ai-lab/SIRIN.

发表机构

• Sber AI Lab（Sber AI实验室）

机构由 AI 辅助整理，请以论文原文为准。