arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.16632cs.CVcs.AI

REEL:学习检索、评估与演化视觉记忆以实现任意长时序多镜头视频生成

REEL: Learning to Retrieve, Evaluate, and Evolve Visual Memory for Any Long-Horizon Multi-Shot Video Generation

University of Science and Technology of China(中国科学技术大学);JD Explore Academy(京东探索研究院)

Hang Xu, Guohui Zhang, Xiaoxiao Ma, Hu Yu, Jie Huang, Jiayin Xiao, Lin Song, Haoyang Huang, Nan Duan, Feng Zhao

AI总结:

本研究提出通用视觉记忆规划器REEL,构建相关数据集并分三阶段训练,在多基准测试中显著提升跨镜头视频生成的一致性。

摘要:

基于记忆的长视频生成旨在将用户控制与长程视觉一致性相结合,我们将该过程组织为规划器、适配器和生成器三个功能组件。代表性多镜头系统将记忆与生成器特定接口耦合,或使用预定义智能体工作流,而非针对跨镜头目标独立训练记忆策略。我们提出REEL,一种通用视觉记忆规划器,通过统一工具接口,REEL学习检索参考、创建缺失视觉证据并更新持久记忆;后端特定适配器将其证据计划转换为支持的条件输入,使同一策略可指导不同黑盒视频生成器。为训练REEL,我们构建两个高质量数据集REEL-SFT-4k和REEL-RL-1k,涵盖多样交互模式与提示类型,训练分三个阶段:监督微调、单镜头组相对策略优化(GRPO)及多镜头GRPO;多镜头GRPO将策略生成的记忆跨镜头传递,使后续结果指导早期参考选择与记忆更新。REEL-Bench在纯文本请求和定时参考图像上传下测试身份保留与预期状态变化,在JoyAI-Echo~1.5和MiniMax-H3上,相较于无记忆基线,REEL在ST-Bench上的跨镜头一致性分别提升15.1%和20.7%,在我们提出的REEL-Bench上的Core-100指标分别提升2.41和8.32个点。

英文摘要:

Memory-based long-video generation aims to combine user control with long-range visual consistency. We organize this process into three functional components: a planner, an adapter, and a generator. Representative multi-shot systems couple memory to generator-specific interfaces or use predefined agent workflows, rather than independently training a memory policy for cross-shot objectives. We introduce REEL, a general-purpose visual-memory planner. Through a unified tool interface, REEL learns to retrieve references, create missing visual evidence, and update persistent memory. Backend-specific adapters translate its evidence plans into supported conditioning inputs, allowing the same policy to guide different black-box video generators. To train REEL, we construct two high-quality datasets, REEL-SFT-4k and REEL-RL-1k, covering diverse interaction patterns and prompt types. Training proceeds in three stages: supervised fine-tuning, single-shot Group Relative Policy Optimization (GRPO), and multi-shot GRPO. Multi-shot GRPO carries policy-generated memory across consecutive shots, so later outcomes guide earlier reference selection and memory updates. REEL-Bench tests identity preservation and intended state changes under text-only requests and timed reference-image uploads. On JoyAI-Echo~1.5 and MiniMax-H3, REEL improves cross-shot consistency on ST-Bench by 15.1\% and 20.7\%, and Core-100 on our proposed REEL-Bench by 2.41 and 8.32 points, respectively, over the no-memory baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑