arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.18092cs.AI
收藏

LifeSearch:面向真实世界生活与生产力的深度研究智能体基准测试

LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity

Tencent(腾讯)

Yibo Wang, Senjie Jin, Yansong Ning, Xinbing Liang, Haotian Luo, Jiazheng Zhang, Yuxuan He, Will Wu, Clarenceai

AI总结:

该研究推出LifeSearch基准,含203项真实生活与生产力任务,经4500小时标注构建细粒度评分标准,评估12个DR智能体,发现最佳者仅得63.4分,验证其评估框架的有效性。

摘要:

深度研究(DR)智能体通过多步推理、长周期网络搜索及证据合成来解决开放式信息需求。然而,现有基准主要聚焦于学术研究或合成任务,未充分评估DR智能体在真实世界生活及专业生产力场景中的能力。为填补这一空白,我们推出LifeSearch,这一具有挑战性的基准包含203项中英文任务,源自7个具体生活与生产力领域的真实用户查询种子。我们投入超过4500小时的人工标注,构建了4000余条由专家撰写的细粒度评分标准,涵盖报告内容的覆盖度与准确性标准,以及指定数值、单位、日期和URL的具体标准。我们在统一评估框架下对12个先进DR智能体开展了广泛实验,系统分析其性能模式与失败模式。研究发现,即便是表现最佳的前沿智能体GPT-5.6 Sol,得分也仅为63.4%,在现实生活与专业DR任务上仍有巨大提升空间。我们通过对比候选大语言模型(LLM)评判器与人工评分标准标签来选择评分配置,人工排序研究显示,基准总分与人工判断具有强一致性(斯皮尔曼相关系数ρ=0.89,肯德尔相关系数τ=0.81)。LifeSearch为推进DR智能体向真实世界生活与生产力场景发展提供了严谨的评估框架与基础。

英文摘要:

Deep research (DR) agents address open-ended information needs through multi-step reasoning, long-horizon web search, and evidence synthesis. However, existing benchmarks primarily focus on academic research or synthetic tasks, leaving DR agents' capabilities in real-world life and professional productivity settings insufficiently evaluated. To bridge this gap, we introduce LifeSearch, a challenging benchmark comprising 203 Chinese and English tasks derived from real-user query seeds across 7 concrete life and productivity domains. With more than 4,500 hours of human annotation, we construct over 4,000 expert-written, fine-grained rubrics, covering both report-content criteria for coverage and accuracy and concrete criteria specifying values, units, dates, and URLs. We conduct extensive experiments on 12 advanced DR agents under a unified evaluation harness, systematically analyzing their performance patterns and failure modes. We find that even the best-performing frontier agent, GPT-5.6 Sol, achieves a score of only 63.4%, leaving substantial room for improvement on realistic life and professional DR tasks. We select the grading configuration by comparing candidate LLM judges against human rubric labels, and a human ranking study shows strong alignment between aggregate benchmark scores and human judgments (Spearman's $ρ= 0.89$, Kendall's $τ= 0.81$). LifeSearch provides a rigorous evaluation framework and foundation for advancing DR agents toward real-world life and productivity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑