图像与句子：扩展交错指令以实现统一的视觉生成
Images in Sentences: Scaling Interleaved Instructions for Unified Visual Generation
AI总结：
本文提出INSET模型，通过将图像作为文本指令中的原生词汇，解决复杂交错指令下的视觉生成问题，提升多图像一致性和文本对齐性能。
摘要：
尽管多模态语言模型的最新进展使从多图像指令生成图像成为可能，现有方法在处理复杂交错指令时表现不佳。这种限制源于当前范式中图像与文本的结构分离，迫使模型必须处理长距离依赖关系以匹配描述与视觉目标。为了解决这些挑战，我们提出了INSET，一个统一生成模型，能够无缝地将图像嵌入到文本指令中。通过将视觉特征直接定位到其对应的语义槽位，INSET利用Transformer的上下文局部性进行精确的对象绑定，有效地将图像视为密集、表达丰富的语言标记。此外，我们引入了一个可扩展的数据引擎，从标准图像和视频数据集中合成1500万高质量的交错样本，利用VLMs和LLMs构建丰富且长周期的序列。在InterleaveBench上的评估结果表明，INSET在多图像一致性和文本对齐方面显著优于现有最先进方法，随着输入复杂性的增加，性能差距扩大。除了标准生成外，我们的方法还自然扩展到多模态图像编辑，将视觉内容作为指令的一部分，以促进高度表达性和创造性的视觉操作。
英文摘要：
While recent advancements in multimodal language models have enabled image generation from expressive multi-image instructions, existing methods struggle to maintain performance under complex interleaved instructions. This limitation stems from the structural separation of images and text in current paradigms, which forces models to bridge difficult long-range dependencies to match descriptions with visual targets. To address these challenges, we propose \texttt{I}mages i\texttt{N} \texttt{SE}n\texttt{T}ences (\textit{a.k.a}, INSET), a unified generation model that seamlessly embeds images as native vocabulary within textual instructions. By positioning visual features directly at their corresponding semantic slots, INSET leverages the contextual locality of transformers for precise object binding, effectively treating images as dense, expressive language tokens. Furthermore, we introduce a scalable data engine that synthesizes 15M high-quality interleaved samples from standard image and video datasets, utilizing VLMs and LLMs to construct rich, long-horizon sequences. Evaluation results on InterleaveBench demonstrate that INSET significantly outperforms state-of-the-art methods in multi-image consistency and text alignment, with performance gaps widening as input complexity increases. Beyond standard generation, our approach inherently extends to multimodal image editing, integrating visual content as part of the instruction to facilitate highly expressive and creative visual manipulations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
