Auto-BenchmarkCard：自动化生成基准测试文档
Auto-BenchmarkCard: Automated Synthesis of Benchmark Documentation
AI总结：
Auto-BenchmarkCard通过多智能体数据提取与大语言模型合成，自动化生成准确的AI基准测试文档，提升基准测试的透明性和可比性。
摘要：
我们提出了Auto-BenchmarkCard，一种生成经过验证的AI基准测试描述的工作流程。基准测试文档通常不完整或不一致，使跨任务或领域的基准测试解释和比较变得困难。Auto-BenchmarkCard通过结合来自异构源（例如Hugging Face、Unitxt、学术论文）的多智能体数据提取与大语言模型驱动的合成来解决这一差距。验证阶段通过FactReasoner工具使用原子蕴含评分来评估事实准确性。该工作流程有潜力促进AI基准测试报告的透明性、可比性和可重用性，使研究人员和实践者能够更好地导航和评估基准测试选择。
英文摘要：
We present Auto-BenchmarkCard, a workflow for generating validated descriptions of AI benchmarks. Benchmark documentation is often incomplete or inconsistent, making it difficult to interpret and compare benchmarks across tasks or domains. Auto-BenchmarkCard addresses this gap by combining multi-agent data extraction from heterogeneous sources (e.g., Hugging Face, Unitxt, academic papers) with LLM-driven synthesis. A validation phase evaluates factual accuracy through atomic entailment scoring using the FactReasoner tool. This workflow has the potential to promote transparency, comparability, and reusability in AI benchmark reporting, enabling researchers and practitioners to better navigate and evaluate benchmark choices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
