arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

ENTLORE:面向企业问答的潜在组织推理的图基准

ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answering

Akrin Zheng, Alexander Wu, Alaia Liu

arXiv 2608.10679首次发表:更新:

AI 中文总结

该研究提出基于图的基准构建框架ENTLORE,针对企业问答的潜在组织推理问题,在56种配置上评估其效果,发现企业问答还依赖隐含组织关系的可用性。

AI 中文摘要

企业问答被定义为检索内部文档并生成基于事实的答案。然而,常规企业记录是工作副产品,其中所需的组织关系在异构来源间保持隐含。现有基准提供了真实的多源证据,但通常实现预定义的答案路径,因此测试的是陈述事实的组合,而非恢复语料库中缺失的目标关系,我们将后者能力称为潜在组织推理。我们引入ENTLORE,一个基于图的基准构建框架,该框架从常规文档、权威组织表和操作记录中重构经审计的企业世界。版本化的组织约定在真值图中验证派生关系,以提供完整的标准答案和证明证书。对齐的匿名版本仅公开文档语料库, withheld 私有结构和目标关系。ENTLORE包含来自三种来源类型的2341份文档,以及907个问题,涵盖显式查找、跨源组合和潜在组织推理,在56种模型和访问配置上进行评估。将发布的世界构造为诱导实体图或可导航知识库可获得最强的可部署结果。然而,即使提供标准答案文档,仍有30.4%的潜在问题未得到解答,而显式问题和组合问题的未解答比例分别为12.6%和6.2%。因此,企业问答不仅取决于文档召回,还取决于隐含的组织关系是否可用。该基准、数据和代码可在ENTLORE公开获取。

英文摘要

Enterprise question answering is framed as retrieving internal documents and generating grounded answers. Routine enterprise records, however, are work by-products in which required organizational relations remain implicit across heterogeneous sources. Existing benchmarks provide realistic multi-source evidence, but often materialize a predefined answer path and therefore test the composition of stated facts rather than recovery of a target relation absent from the corpus. We call the latter capability latent organizational reasoning. We introduce ENTLORE, a graph-grounded benchmark construction framework that reconstructs an audited enterprise world from routine documents, authoritative organizational tables, and operational records. Versioned organizational conventions certify derived relations in a truth graph, enabling complete golden answers and proof certificates. The aligned anonymized release exposes only the document corpus while withholding private structure and target relations. ENTLORE contains 2,341 documents from three source types and 907 questions spanning explicit lookup, cross-source composition, and latent organizational reasoning, evaluated across 56 model and access configurations. Structuring the released world as an induced entity graph or navigable knowledge base gives the strongest deployable results. Yet supplying gold documents still leaves 30.4% of latent questions unanswered, versus 12.6% and 6.2% for explicit and compositional questions. Enterprise QA therefore depends not only on document recall, but also on whether implicit organizational relations become usable. The benchmark, data, and code are publicly available at https://github.com/scitix/entlore .

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑