arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.11685cs.LGcs.AI
收藏

DRACO:跨领域基准测试用于深度研究的准确性、完整性和客观性

DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity

Joey Zhong, Hao Zhang, Clare Southern, Jeremy Yang, Thomas Wang, Kate Jung, Shu Zhang, Denis Yarats, Johnny Ho, Jerry Ma

AI总结:

DRACO是一个跨领域基准测试,用于评估深度研究任务的准确性、完整性和客观性,通过四个维度评分系统确保任务的客观评估。

摘要:

我们提出了DRACO(深度研究准确性、完整性和客观性),一个复杂的深度研究任务基准测试。这些任务涵盖了10个领域,并利用来自40个国家的信息源,起源于大规模深度研究系统中匿名的现实使用模式。任务是从去标识化的Perplexity深度研究请求数据集中采样,然后经过过滤和增强,以确保任务是匿名的、开放性的和复杂的,可以客观评估,并代表了广泛的真实世界深度研究使用案例。输出根据四个维度进行评分:事实准确性(准确性)、分析的广度和深度(包括完整性)、呈现质量(包括客观性)以及引用质量。DRACO在https://hf.co/datasets/perplexity-ai/draco上公开可用。

英文摘要:

We present DRACO (Deep Research Accuracy, Completeness, and Objectivity), a benchmark of complex deep research tasks. These tasks, which span 10 domains and draw on information sources from 40 countries, originate from anonymized real-world usage patterns within a large-scale deep research system. Tasks are sampled from a de-identified dataset of Perplexity Deep Research requests, then filtered and augmented to ensure that the tasks are anonymized, open-ended and complex, objectively evaluable, and representative of the broad scope of real-world deep research use cases. Outputs are graded against task-specific rubrics along four dimensions: factual accuracy (accuracy), breadth and depth of analysis (including completeness), presentation quality (including objectivity), and citation quality. DRACO is publicly available at https://hf.co/datasets/perplexity-ai/draco.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑