arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.04733cs.AI
收藏

面向沉浸式视频角色扮演的奖励分解强化学习

Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing

Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学);University of California, San Diego(加州大学圣地亚哥分校);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);School of Information Engineering, Beijing Institute of Graphic Communication(北京印刷学院信息工程学院);Ant International, Ant Group(蚂蚁集团国际部);Independent Researcher(独立研究者)

Miao Wang, Yuling Shi, Yijiang Li, Yeheng Chen, Xiaodong Gu, Bin Li, Bo Gao, Jun Wang, Zengxin Han, Jingtong Wu, Yaduan Ruan

AI总结:

提出EBM-RL框架,通过奖励分解的强化学习优化视频角色扮演中的视觉感知、推理与生成过程,提升场景一致性与角色真实性。

摘要:

基于文本的角色扮演模型可以模仿角色风格,但通常难以捕捉场景氛围和不断变化的紧张感,而这些对于VR游戏和互动叙事等沉浸式应用至关重要。我们研究视频驱动的角色扮演对话,并引入EBM-RL(眼-脑-口强化学习),一种解耦的GRPO框架,将观察(<perception>)、推理(<think>)和话语生成(<answer>)分离。该设计模仿人类的“看-思-说”过程,使模型在推理和响应生成之前能够基于视觉感知进行对话。为了优化这一“看-思-说”过程,EBM-RL集成了针对场景-文本对齐、感知-认知效用、答案忠实度和格式一致性的互补奖励。大量实验表明,在我们的沉浸式角色扮演基准测试中,EBM-RL显著优于纯文本角色扮演基线和更大规模的视觉语言模型,提高了视觉-氛围一致性和角色真实性。此外,EBM-RL在无需额外微调的情况下,展现出对域外VideoQA基准的强零样本迁移能力。我们还发布了一个用于视频驱动角色扮演对话的开源数据集。

英文摘要:

Text-based role-playing models can imitate character styles, but often fail to capture scene atmosphere and evolving tension, which are crucial for immersive applications such as VR games and interactive narratives. We study video-grounded role-playing dialogue and introduce EBM-RL (Eye--Brain--Mouth Reinforcement Learning), a decoupled GRPO-based framework that separates observation (<perception>), reasoning (<think>), and utterance generation (<answer>). This design mimics the human See-Think-Speak process, enabling the model to ground dialogue in visual perception before reasoning and response generation. To optimize this See-Think-Speak process, EBM-RL integrates complementary rewards for scene--text alignment, perceptual--cognitive utility, answer faithfulness, and format consistency. Extensive experiments show that EBM-RL substantially outperforms text-only role-playing baselines and larger-scale vision-language models on our immersive role-playing benchmark, improving both visual-atmosphere consistency and character authenticity. Moreover, EBM-RL demonstrates strong zero-shot transfer to out-of-domain VideoQA benchmarks without additional fine-tuning. We also release an open-source dataset for video-grounded role-playing dialogue.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑