超越行到推理:面向多模态电子表格理解与编辑的智能检索
Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editing
AI总结:
BRTR通过迭代工具调用框架实现多模态电子表格的端到端理解与编辑,取得领先性能。
摘要:
最近多模态检索增强生成(RAG)的进展使大型语言模型(LLMs)能够分析包含数百万个单元格、跨表依赖关系和嵌入式视觉元素的企业电子表格工作簿。然而,最先进的方法通过单次检索排除了关键上下文,通过压缩丢失数据分辨率,并通过简单全上下文注入超出LLM上下文窗口,从而防止了对复杂企业工作簿的可靠多步推理。我们引入了
英文摘要:
Recent advances in multimodal Retrieval-Augmented Generation (RAG) enable Large Language Models (LLMs) to analyze enterprise spreadsheet workbooks containing millions of cells, cross-sheet dependencies, and embedded visual artifacts. However, state-of-the-art approaches exclude critical context through single-pass retrieval, lose data resolution through compression, and exceed LLM context windows through naive full-context injection, preventing reliable multi-step reasoning over complex enterprise workbooks. We introduce Beyond Rows to Reasoning (BRTR), a multimodal agentic framework for spreadsheet understanding that replaces single-pass retrieval with an iterative tool-calling loop, supporting end-to-end Excel workflows from complex analysis to structured editing. Supported by over 200 hours of expert human evaluation, BRTR achieves state-of-the-art performance across three frontier spreadsheet understanding benchmarks, surpassing prior methods by 25 percentage points on FRTR-Bench, 7 points on SpreadsheetLLM, and 32 points on FINCH. We evaluate five multimodal embedding models, identifying NVIDIA NeMo Retriever 1B as the top performer for mixed tabular and visual data, and vary nine LLMs. Ablation experiments confirm that the planner, retrieval, and iterative reasoning each contribute substantially, and cost analysis shows GPT-5.2 achieves the best efficiency-accuracy trade-off. Throughout all evaluations, BRTR maintains full auditability through explicit tool-call traces.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。