检索增强结构化生成：将商业文档信息抽取作为工具使用
Retrieval Augmented Structured Generation: Business Document Information Extraction As Tool Use
AI总结：
本文将商业文档信息抽取建模为工具使用问题，提出RASG框架，使LLM在KIE和LIR任务上超越LMM，并设计了更贴合实际的新指标GLIRM及无需视觉编码器的边界框反算启发式算法。
摘要：
商业文档信息抽取（BDIE）是将一团非结构化信息（原始文本、扫描文档等）转换为下游系统能够解析和使用的结构化格式的问题。它有两个主要任务：关键信息抽取（KIE）和行项目识别（LIR）。在本文中，我们认为BDIE最好被建模为工具使用问题，其中工具就是这些下游系统。然后我们提出了检索增强结构化生成（RASG），这是一种用于BDIE的新型通用框架，在BDIE基准测试的KIE和LIR任务上均取得了最先进（SOTA）的结果。本文的贡献有三方面：（1）我们通过消融基准测试表明，带有RASG的大型语言模型（LLM）在BDIE基准测试上已经与当前的SOTA大型多模态模型（LMM）具有竞争力，甚至在没有RASG的情况下超越了它们。（2）我们提出了一类新的行项目识别指标，即通用行项目识别指标（GLIRM），与现有指标（如ANLS*、DocILE和GriTS）相比，它更符合实际的BDIE用例。（3）我们提供了一种启发式算法，用于反算预测的行项目和表格的边界框，而无需视觉编码器。最后，我们声称，虽然LMM有时可能提供边际性能收益，但考虑到BDIE的实际应用和约束，LLM + RASG通常更优越。
英文摘要：
Business Document Information Extraction (BDIE) is the problem of transforming a blob of unstructured information (raw text, scanned documents, etc.) into a structured format that downstream systems can parse and use. It has two main tasks: Key-Information Extraction (KIE) and Line Items Recognition (LIR). In this paper, we argue that BDIE is best modeled as a Tool Use problem, where the tools are these downstream systems. We then present Retrieval Augmented Structured Generation (RASG), a novel general framework for BDIE that achieves state of the art (SOTA) results on both KIE and LIR tasks on BDIE benchmarks. The contributions of this paper are threefold: (1) We show, with ablation benchmarks, that Large Language Models (LLMs) with RASG are already competitive with or surpasses current SOTA Large Multimodal Models (LMMs) without RASG on BDIE benchmarks. (2) We propose a new metric class for Line Items Recognition, General Line Items Recognition Metric (GLIRM), that is more aligned with practical BDIE use cases compared to existing metrics, such as ANLS*, DocILE, and GriTS. (3) We provide a heuristic algorithm for backcalculating bounding boxes of predicted line items and tables without the need for vision encoders. Finally, we claim that, while LMMs might sometimes offer marginal performance benefits, LLMs + RASG is oftentimes superior given real-world applications and constraints of BDIE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
