Paper2Video：从科学论文自动生成视频
Paper2Video: Automatic Video Generation from Scientific Papers
AI总结：
针对学术演示视频制作耗时的问题，本文提出Paper2Video基准和PaperTalker多智能体框架，通过树搜索布局优化等实现自动生成，实验证明其更忠实且信息丰富。
摘要：
学术演示视频已成为研究交流的重要媒介，然而制作此类视频仍然高度劳动密集，通常需要数小时的幻灯片设计、录制和编辑才能生成一个2至10分钟的短视频。与自然视频不同，演示视频生成面临独特的挑战：输入来自研究论文，包含密集的多模态信息（文本、图表、表格），并且需要协调多个对齐的通道，如幻灯片、字幕、语音和真人演讲者。为应对这些挑战，我们提出了Paper2Video，这是首个包含101篇研究论文及其作者创建的演示视频、幻灯片和演讲者元数据的基准数据集。我们进一步设计了四个定制的评估指标——Meta Similarity、PresentArena、PresentQuiz和IP Memory——来衡量视频向观众传达论文信息的程度。在此基础上，我们提出了PaperTalker，这是首个用于学术演示视频生成的多智能体框架。它通过一种新颖的高效树搜索视觉选择、光标定位、字幕生成、语音合成和说话人头像渲染，将幻灯片生成与有效的布局优化相结合，同时并行化逐张幻灯片的生成以提高效率。在Paper2Video上的实验表明，我们方法生成的演示视频比现有基线更忠实且信息更丰富，为自动化且可立即使用的学术视频生成迈出了实际的一步。我们的数据集、智能体和代码可在https://github.com/showlab/Paper2Video获取。
英文摘要：
Academic presentation videos have become an essential medium for research communication, yet producing them remains highly labor-intensive, often requiring hours of slide design, recording, and editing for a short 2 to 10 minutes video. Unlike natural video, presentation video generation involves distinctive challenges: inputs from research papers, dense multi-modal information (text, figures, tables), and the need to coordinate multiple aligned channels such as slides, subtitles, speech, and human talker. To address these challenges, we introduce Paper2Video, the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata. We further design four tailored evaluation metrics--Meta Similarity, PresentArena, PresentQuiz, and IP Memory--to measure how videos convey the paper's information to the audience. Building on this foundation, we propose PaperTalker, the first multi-agent framework for academic presentation video generation. It integrates slide generation with effective layout refinement by a novel effective tree search visual choice, cursor grounding, subtitling, speech synthesis, and talking-head rendering, while parallelizing slide-wise generation for efficiency. Experiments on Paper2Video demonstrate that the presentation videos produced by our approach are more faithful and informative than existing baselines, establishing a practical step toward automated and ready-to-use academic video generation. Our dataset, agent, and code are available at https://github.com/showlab/Paper2Video.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
