TRIAGE：可信检索工具与图评估
TRIAGE: Trustworthy Retrieval Instrumentation And Graph Evaluation
AI总结：
研究基于LLM驱动自动构建的知识图谱评估问题，引入TRIAGE框架，通过特定阶段指标对知识图谱构建与使用各阶段评估，可定位失败环节并给出改进方向。
摘要：
支撑基于图的检索增强生成（Graph-RAG）的知识图谱（KGs）越来越多地由LLM驱动提取自动构建而非专家策划。恰当评估需对提取、图构建和推理等所有相关阶段进行连贯监测以定位故障。我们引入TRIAGE，一个用于自动化、文档基础的图-RAG的阶段感知监测框架，不仅询问底层图是否可信，还询问查询它的成本。TRIAGE为三个阶段附加特定阶段、可独立解释的指标……
英文摘要：
Knowledge graphs (KGs) that underpin Graph-based Retrieval-Augmented Generation (Graph-RAG) are increasingly built automatically by LLM-driven extraction rather than curated by experts. Proper evaluation would require instrumenting all pertinent stages: extraction, graph construction, and inference, coherently enough to localize failures, so that a failure at one stage is not discovered as a wrong answer at the end. We introduce TRIAGE, a stage-aware instrumentation framework for automated, document-grounded graph-RAG that asks not only whether the underlying graph can be trusted but at what cost it can be queried. TRIAGE attaches stage-specific, independently interpretable metrics to three stages: the KG Implementation (triple confidence, source coverage, and schema and canonicalization checks), the KG Validation by expert (graph-level structural quality, with correctness and completeness computed only as offline calibration when a reference is available), and the KG Usage (retrieval coverage, faithfulness, and retrieval cost); the deployed metrics need no gold annotations, the gold-requiring ones serving only as offline calibration. At usage time these metrics form a diagnostic chain of necessary conditions whose first broken link localizes the failure, and the diagnosis maps to the stage levers that can remedy it: extraction, graph and schema, or retrieval. TRIAGE is a theoretical framework with a proof of concept and a reproducible evaluation protocol.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
