ChatUMM: 面向对话式交错生成的鲁棒上下文追踪
ChatUMM: Robust Context Tracking for Conversational Interleaved Generation
AI总结:
提出ChatUMM,一种通过交错多轮训练策略和系统化对话数据合成流水线实现鲁棒上下文追踪的对话式统一多模态模型,在视觉理解和指令引导编辑基准上达到开源模型最优性能。
摘要:
统一多模态模型(UMMs)取得了显著进展,但仍受限于单轮交互范式,实际上作为独立请求的求解器而非连续对话中的助手。为弥合这一差距,我们提出了ChatUMM。作为一种对话式统一模型,它擅长鲁棒的上下文追踪以维持交错多模态生成。ChatUMM的能力源于两个关键创新:一种交错多轮训练策略,将序列化的文本-图像流建模为连续的对话流;以及一个系统化的对话数据合成流水线。该流水线通过三个渐进阶段将多样化的标准单轮数据集转化为流畅的对话:构建基本有状态对话,通过带有历史依赖查询重写的“干扰”轮次强制执行长程依赖解析,以及合成自然交错的多模态响应。大量评估表明,ChatUMM在视觉理解和指令引导编辑基准上达到了开源统一模型中的最先进性能,同时在文本到图像生成中保持了有竞争力的保真度。值得注意的是,ChatUMM在复杂多轮场景中展现出卓越的鲁棒性,确保了流畅、上下文感知的对话。
英文摘要:
Unified multimodal models (UMMs) have achieved remarkable progress yet remain constrained by a single-turn interaction paradigm, effectively functioning as solvers for independent requests rather than assistants in continuous dialogue. To bridge this gap, we present ChatUMM. As a conversational unified model, it excels at robust context tracking to sustain interleaved multimodal generation. ChatUMM derives its capabilities from two key innovations: an interleaved multi-turn training strategy that models serialized text-image streams as a continuous conversational flow, and a systematic conversational data synthesis pipeline. This pipeline transforms a diverse set of standard single-turn datasets into fluid dialogues through three progressive stages: constructing basic stateful dialogues, enforcing long-range dependency resolution via ``distractor'' turns with history-dependent query rewriting, and synthesizing naturally interleaved multimodal responses. Extensive evaluations demonstrate that ChatUMM achieves state-of-the-art performance among open-source unified models on visual understanding and instruction-guided editing benchmarks, while maintaining competitive fidelity in text-to-image generation. Notably, ChatUMM exhibits superior robustness in complex multi-turn scenarios, ensuring fluid, context-aware dialogues.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。