arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.07720cs.AI
收藏

走向知识导向的深度研究:框架与基准

Towards Knowledgeable Deep Research: Framework and Benchmark

State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所人工智能安全国家重点实验室);National University of Singapore(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学)
Wenxuan Liu, Zixuan Li, Long Bai, Chunmao Zhang, Fenghui Zhang, Zhuo Chen, Wei Li, Yuxin Zuo, Fei Wang, Bingbing Xu, Xuhui Jiang, Jin Zhang, Xiaolong Jin, Jiafe… 展开作者

Wenxuan Liu, Zixuan Li, Long Bai, Chunmao Zhang, Fenghui Zhang, Zhuo Chen, Wei Li, Yuxin Zuo, Fei Wang, Bingbing Xu, Xuhui Jiang, Jin Zhang, Xiaolong Jin, Jiafeng Guo, Tat-Seng Chua, Xueqi Cheng

AI总结:

本文提出知识导向深度研究(KDR)任务,设计混合知识分析框架(HKA)并构建KDR-Bench基准,通过结构化与非结构化知识生成多模态报告,实验表明HKA在通用和知识导向指标上优于现有方法,且在视觉增强指标上超越Gemini DR代理。

摘要:

深度研究(DR)要求LLM代理自主执行多步骤信息检索、处理和推理以生成综合报告。与现有研究主要关注非结构化网络内容不同,更具挑战性的DR任务应利用结构化知识提供坚实的数据基础,促进定量计算并实现深入分析。本文将此新任务称为知识导向深度研究(KDR),要求DR代理生成包含结构化和非结构化知识的报告。此外,我们提出混合知识分析框架(HKA),一种多代理架构,能够对两种知识进行推理并整合文本、图表和表格生成连贯的多模态报告。关键设计是结构化知识分析器,利用编码和视觉-语言模型生成图表、表格及相应洞察。为支持系统评估,我们构建KDR-Bench,涵盖9个领域,包含41个专家级问题,并整合大量结构化知识资源(例如1,252张表格)。我们进一步标注每个问题的主要结论和关键点,并提出三种评估指标类别,包括通用型、知识导向型和视觉增强型。实验结果表明,HKA在通用型和知识导向型指标上优于大多数现有DR代理,并在视觉增强型指标上超越Gemini DR代理,凸显其在深度、结构化知识分析中的有效性。最后,我们希望本工作能为DR代理中的结构化知识分析提供新基础,并促进未来多模态DR研究。

英文摘要:

Deep Research (DR) requires LLM agents to autonomously perform multi-step information seeking, processing, and reasoning to generate comprehensive reports. In contrast to existing studies that mainly focus on unstructured web content, a more challenging DR task should additionally utilize structured knowledge to provide a solid data foundation, facilitate quantitative computation, and lead to in-depth analyses. In this paper, we refer to this novel task as Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge. Furthermore, we propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that reasons over both kinds of knowledge and integrates the texts, figures, and tables into coherent multimodal reports. The key design is the Structured Knowledge Analyzer, which utilizes both coding and vision-language models to produce figures, tables, and corresponding insights. To support systematic evaluation, we construct KDR-Bench, which covers 9 domains, includes 41 expert-level questions, and incorporates a large number of structured knowledge resources (e.g., 1,252 tables). We further annotate the main conclusions and key points for each question and propose three categories of evaluation metrics including general-purpose, knowledge-centric, and vision-enhanced ones. Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics, and even surpasses the Gemini DR agent on vision-enhanced metrics, highlighting its effectiveness in deep, structure-aware knowledge analysis. Finally, we hope this work can serve as a new foundation for structured knowledge analysis in DR agents and facilitate future multimodal DR studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑