NExT-OMNI：基于离散流匹配的任意到任意全模态基础模型
NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching
AI总结：
NExT-OMNI利用离散流匹配实现任意到任意的全模态统一建模，在生成、理解、多轮交互及跨模态检索上超越先前统一模型，并开源代码与模型。
摘要：
下一代多模态基础模型能够实现任意到任意的跨模态生成和多轮交互，将成为通用人工智能系统的核心组件，在人机交互中发挥关键作用。然而，现有的大多数多模态模型仍受限于自回归架构，其固有缺陷阻碍了理解与生成能力的均衡整合。尽管已有研究探索了混合和解耦策略，以在统一框架内分别处理这些任务，但其冗余且非集成的设计限制了它们在更广泛场景（如跨模态检索）中的适用性。在本工作中，我们提出了NExT-OMNI，一个开源的、通过离散流范式实现统一建模的全模态基础模型。通过利用度量诱导的概率路径和动力学最优速度，NExT-OMNI原生支持任意到任意的理解与生成，并具有更高的响应效率，同时通过简洁的统一表示而非任务解耦设计，支持更广泛的应用场景。NExT-OMNI在大规模交错文本、图像、视频和音频数据上进行训练，在多模态生成和理解基准上取得了具有竞争力的性能，同时在多轮多模态交互和跨模态检索方面优于先前的统一模型，凸显了其作为下一代多模态基础模型的架构优势。为了推动进一步研究，我们发布了训练细节、数据协议，并开源了代码和模型检查点。
英文摘要：
Next-generation multimodal foundation models capable of any-to-any cross-modal generation and multi-turn interaction will serve as core components of artificial general intelligence systems, playing a pivotal role in human-machine interaction. However, most existing multimodal models remain constrained by autoregressive architectures, whose inherent limitations prevent a balanced integration of understanding and generation capabilities. Although hybrid and decoupling strategies have been explored to address these tasks within unified frameworks separately, their redundant, non-integrated designs limit their applicability to broader scenarios, such as cross-modal retrieval. In this work, we introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms. By leveraging metric-induced probability paths and kinetic optimal velocities, NExT-OMNI natively supports any-to-any understanding and generation with enhanced response efficiency, while enabling broader application scenarios through concise unified representations rather than task-decoupled designs. Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks, while outperforming prior unified models in multi-turn multimodal interaction and cross-modal retrieval, highlighting its architectural advantages as a next-generation multimodal foundation model. To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
