arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.06757cs.CV
收藏

FlowInOne:将多模态生成统一为图像输入、图像输出的流匹配

FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching

University of Electronic Science and Technology of China(电子科技大学);Central South University(中南大学);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Microsoft(微软)

Junchao Yi, Rui Zhao, Jiahao Tang, Weixian Lei, Linjie Li, Qisheng Su, Zhengyuan Yang, Lijuan Wang, Xiaofeng Zhu, Alex Jinpeng Wang

AI总结:

本文提出FlowInOne框架,将多模态生成统一为纯视觉流匹配,通过视觉提示消除跨模态对齐瓶颈,实现文本到图像生成、布局引导编辑和视觉指令遵循的统一。

摘要:

多模态生成长期由文本驱动管道主导,语言决定视觉但无法在其中推理或创造。本文挑战这一范式,探讨是否所有模态,包括文本描述、空间布局和编辑指令,能统一为单一视觉表示。我们提出FlowInOne框架,将多模态生成重新表述为纯视觉流,将所有输入转换为视觉提示,使图像输入、图像输出流程由单一流匹配模型控制。这种以视觉为中心的表述自然消除了跨模态对齐瓶颈、噪声调度和任务特定的架构分支,将文本到图像生成、布局引导编辑和视觉指令遵循统一在一致的范式下。为此,我们引入VisPrompt-5M,一个包含500万对视觉提示的大型数据集,涵盖物理感知力动态和轨迹预测等多样化任务,以及VP-Bench,一个严格筛选的基准测试,评估指令忠实度、空间精度、视觉真实性和内容一致性。大量实验表明,FlowInOne在所有统一生成任务中均取得最先进的性能,超越开源模型和竞争商业系统,建立了一个全新的完全视觉中心生成模型基础,其中感知与创造共存于单一连续视觉空间中。我们的代码和模型已发布在https://csu-jpg.github.io/FlowInOne.github.io/

英文摘要:

Multimodal generation has long been dominated by text-driven pipelines where language dictates vision but cannot reason or create within it. We challenge this paradigm by asking whether all modalities, including textual descriptions, spatial layouts, and editing instructions, can be unified into a single visual representation. We present FlowInOne, a framework that reformulates multimodal generation as a purely visual flow, converting all inputs into visual prompts and enabling a clean image-in, image-out pipeline governed by a single flow matching model. This vision-centric formulation naturally eliminates cross-modal alignment bottlenecks, noise scheduling, and task-specific architectural branches, unifying text-to-image generation, layout-guided editing, and visual instruction following under one coherent paradigm. To support this, we introduce VisPrompt-5M, a large-scale dataset of 5 million visual prompt pairs spanning diverse tasks including physics-aware force dynamics and trajectory prediction, alongside VP-Bench, a rigorously curated benchmark assessing instruction faithfulness, spatial precision, visual realism, and content consistency. Extensive experiments demonstrate that FlowInOne achieves state-of-the-art performance among open-source models across all unified generation tasks while remaining competitive with leading commercial systems, thereby establishing a new foundation for fully vision-centric generative modeling, in which perception and creation coexist within a unified continuous visual space. Our code and models are released on https://csu-jpg.github.io/FlowInOne.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑