arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.02812cs.AI
收藏

通过无提示协作代理增强自动论文复现

Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agents

University of Science and Technology of China(中国科学技术大学);Meituan(美团)

Zijie Lin, Qilin Cai, Liang Shen, Mingjun Xiao

AI总结:

本文提出一种无提示协作代理框架,通过验证和细化代理提升论文到代码生成的准确性和完整性,实验显示性能提升约15%和13%。

摘要:

自动论文复现已作为一种有前景的方法来加速科学研究,通过多步骤工作流程框架系统地将学术论文转换为可执行代码。然而,现有框架往往缺乏在每个生成步骤中验证和细化输出的机制,或严重依赖人工设计的提示进行自我细化,这限制了它们的适应性和可扩展性。为了解决这些限制,我们提出了一种无提示的协作代理框架,该框架能够自动提高论文到代码生成的质量。我们的方法采用了两个协作代理:一个验证代理,用于检查每个步骤的输出是否满足相应系统提示中指定的要求,另一个细化代理,根据发现的问题修订输出。与以往需要人类专家为每一步设计特定细化提示的方法不同,我们的框架通过仅利用原始系统提示实现了自动验证和改进。我们将我们的协作代理集成到Paper2Code框架中,并在PaperBench Code-Dev和Paper2CodeBench数据集中进行了全面实验。实验结果表明,我们的方法显著提高了复现代码的准确性和完整性,分别比没有我们代理的基线方法提高了约15%和13%。此外,与Self-Refine的对比实验验证了我们的无提示方法在不同数据集上的鲁棒性和一致性。

英文摘要:

Automated paper reproduction has emerged as a promising approach to accelerate scientific research, employing multi-step workflow frameworks to systematically convert academic papers into executable code. However, existing frameworks often lack mechanisms to verify and refine the outputs at each generation step, or rely heavily on manually designed prompts for self-refinement, which limits their adaptability and scalability. To address these limitations, we propose a prompt-free collaborative agent framework that automatically enhances the quality of paper-to-code generation. Our approach employs two collaborative agents: a verification agent that examines whether the outputs at each step satisfy the requirements specified in the corresponding system prompt, and a refinement agent that revises the outputs based on the identified issues. Unlike previous methods that require human experts to craft specific refinement prompts for each step, our framework achieves automatic verification and improvement by leveraging only the original system prompts. We integrate our collaborative agents into the Paper2Code framework and conduct comprehensive experiments on PaperBench Code-Dev and Paper2CodeBench datasets. Experimental results demonstrate that our approach significantly improves the accuracy and completeness of reproduced code, achieving performance gains of approximately 15\% and 13\%, respectively, compared to the baseline without our agents. Furthermore, comparative experiments against Self-Refine validate the robustness and consistency of our prompt-free approach across different datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑