WebWalker:在Web遍历中对LLM进行基准测试
WebWalker: Benchmarking LLMs in Web Traversal
AI总结:
针对传统搜索引擎检索内容浅显限制LLM处理复杂信息的问题,提出WebWalkerQA基准及多智能体框架WebWalker,通过探索-评估范式模拟人类网页导航,实验证明其结合RAG在真实场景中有效。
摘要:
检索增强生成(RAG)在开放域问答任务中表现出色。然而,传统搜索引擎可能检索到浅层内容,限制了LLM处理复杂、多层信息的能力。为解决此问题,我们引入了WebWalkerQA,这是一个旨在评估LLM执行Web遍历能力的基准。它评估LLM系统地遍历网站子页面以提取高质量数据的能力。我们提出了WebWalker,这是一个多智能体框架,通过探索-评估范式模拟类似人类的Web导航。大量实验结果表明,WebWalkerQA极具挑战性,并通过在真实场景中的横向与纵向整合,证明了RAG与WebWalker结合的有效性。
英文摘要:
Retrieval-augmented generation (RAG) demonstrates remarkable performance across tasks in open-domain question-answering. However, traditional search engines may retrieve shallow content, limiting the ability of LLMs to handle complex, multi-layered information. To address it, we introduce WebWalkerQA, a benchmark designed to assess the ability of LLMs to perform web traversal. It evaluates the capacity of LLMs to traverse a website's subpages to extract high-quality data systematically. We propose WebWalker, which is a multi-agent framework that mimics human-like web navigation through an explore-critic paradigm. Extensive experimental results show that WebWalkerQA is challenging and demonstrates the effectiveness of RAG combined with WebWalker, through the horizontal and vertical integration in real-world scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。