arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.28185cs.CV

新时代的视觉生成:从原子映射到智能世界建模的演进

Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling

Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);University of Hong Kong(香港大学);National University of Singapore(新加坡国立大学);University of Waterloo(滑铁卢大学);StepFun;MiroMind;Baidu(百度);Fudan University(复旦大学);Hong Kong University of Science and Technology(香港科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));LMMs-Lab(LMMs实验室)
Keming Wu, Zuhao Yang, Kaichen Zhang, Shizun Wang, Haowei Zhu, Sicong Leng, Zhongyu Yang, Qijie Wang, Sudong Wang, Ziting Wang, Zili Wang, Hui Zhang, Haonan Wan… 展开作者

Keming Wu, Zuhao Yang, Kaichen Zhang, Shizun Wang, Haowei Zhu, Sicong Leng, Zhongyu Yang, Qijie Wang, Sudong Wang, Ziting Wang, Zili Wang, Hui Zhang, Haonan Wang, Hang Zhou, Yifan Pu, Xingxuan Li, Fangneng Zhan, Bo Li, Lidong Bing, Yuxin Song, Ziwei Liu, Wenhu Chen, Jingdong Wang, Xinchao Wang, Xiaojuan Qi, Shijian Lu, Bin Wang

AI总结:

提出五级分类法(原子生成、条件生成、上下文生成、智能生成、世界建模生成),分析流匹配、统一理解与生成模型等关键技术,并指出现有评估高估感知质量而忽视结构、时序和因果缺陷。

摘要:

最近的视觉生成模型在照片级真实感、排版、指令遵循和交互式编辑方面取得了重大进展,但在空间推理、持久状态、长期一致性和因果理解方面仍然存在困难。我们认为,该领域应超越外观合成,转向智能视觉生成:基于结构、动态、领域知识和因果关系的合理视觉内容。为构建这一转变,我们引入了一个五级分类法:原子生成、条件生成、上下文生成、智能生成和世界建模生成,从被动渲染器逐步演进到交互式、智能、世界感知的生成器。我们分析了关键技术驱动因素,包括流匹配、统一理解与生成模型、改进的视觉表示、后训练、奖励建模、数据整理、合成数据蒸馏和采样加速。我们进一步表明,当前的评估往往通过强调感知质量而高估了进展,却忽略了结构、时序和因果缺陷。通过结合基准测试回顾、野外压力测试和专家约束案例研究,本路线图提供了一个以能力为中心的视角,用于理解、评估和推进下一代智能视觉生成系统。

英文摘要:

Recent visual generation models have made major progress in photorealism, typography, instruction following, and interactive editing, yet they still struggle with spatial reasoning, persistent state, long-horizon consistency, and causal understanding. We argue that the field should move beyond appearance synthesis toward intelligent visual generation: plausible visuals grounded in structure, dynamics, domain knowledge, and causal relations. To frame this shift, we introduce a five-level taxonomy: Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation, progressing from passive renderers to interactive, agentic, world-aware generators. We analyze key technical drivers, including flow matching, unified understanding-and-generation models, improved visual representations, post-training, reward modeling, data curation, synthetic data distillation, and sampling acceleration. We further show that current evaluations often overestimate progress by emphasizing perceptual quality while missing structural, temporal, and causal failures. By combining benchmark review, in-the-wild stress tests, and expert-constrained case studies, this roadmap offers a capability-centered lens for understanding, evaluating, and advancing the next generation of intelligent visual generation systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑