arXiv 2605.104452026-05-11cs.CV

Uni-Synergy：通过协作强化学习实现个性化推理的理解与生成弥合

Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning

Peking University（北京大学）；Nanjing University（南京大学）；CUHK（香港中文大学）；Zhongguancun Academy（中关村学院）

Zijun Shen, Sihan Yang, Ruichuan An, Ziyu Guo, Hao Liang, Ming Lu, Renrui Zhang, Wentao Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Sync-R1框架，通过协作强化学习统一优化个性化理解和生成，结合Sync-GRPO和DGS方法提升双任务协同效率，实验证明其在跨任务推理和鲁棒个性化方面的优越性能。

摘要：

统一多模态模型（UMMs）在通用任务中表现出色，但在个性化理解与生成之间的桥梁建设上存在困难。现有方法主要依赖于监督微调的隐式令牌级对齐，无法充分捕捉理解和创造之间的潜在协同效应。本文提出Sync-R1，一种端到端的强化学习框架，通过单一显式推理循环联合优化个性化理解和生成。通过这种统一的反馈过程，Sync-R1使个性化理解引导内容创作，而生成质量反过来在整合的奖励景观中细化理解。为高效协调双任务协同，我们引入Sync-GRPO，一种利用集成奖励系统的强化学习方法。此外，我们提出动态分组缩放（DGS），通过自适应过滤低潜力轨迹来减少梯度方差并加速收敛。为更好地反映现实世界的复杂性，我们引入UnifyBench++，其特征更密集的文本描述和更丰富的用户上下文。实验结果表明，Sync-R1在跨任务推理和鲁棒个性化方面实现了最先进的性能，无需复杂的冷启动过程。代码和UnifyBench++数据集将在https://github.com/arctanxarc/UniCTokens发布。

英文摘要：

Unified Multimodal Models (UMMs) excel in general tasks but struggle to bridge the gap between personalized understanding and generation. Prior works largely rely on implicit token-level alignment via supervised fine-tuning, which fails to fully capture the potential synergy between comprehension and creation. In this work, we propose Sync-R1, an end-to-end reinforcement learning framework that jointly optimizes personalized understanding and generation within a single, explicit reasoning loop. Through this unified feedback process, Sync-R1 enables personalized comprehension to guide content creation, while the resulting generation quality reciprocally refines understanding within an integrated reward landscape. To efficiently orchestrate this dual-task synergy, we introduce Sync-GRPO, a reinforcement learning method utilizing an ensemble reward system. Furthermore, we propose Dynamic Group Scaling (DGS), which adaptively filters low-potential trajectories to reduce gradient variance and accelerate convergence. To better reflect real-world complexity, we introduce UnifyBench++, featuring denser textual descriptions and richer user contexts. Experimental results demonstrate that Sync-R1 achieves state-of-the-art performance, showcasing superior cross-task reasoning and robust personalization without requiring complex cold-start procedures. The code and the UnifyBench++ dataset will be released at: https://github.com/arctanxarc/UniCTokens.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。