VisPlay: 从图像中自我进化视觉-语言模型
VisPlay: Self-Evolving Vision-Language Models from Images
AI总结：
VisPlay通过自我进化强化学习框架，利用未标注图像数据提升视觉-语言模型的推理能力，实现多模态智能的可扩展发展。
摘要：
强化学习（RL）为提升视觉-语言模型（VLMs）在复杂推理任务上的性能提供了系统性的框架。然而，现有RL方法往往依赖于人工标注的标签或任务特定的启发式方法来定义可验证的奖励，这两种方法都成本高且难以扩展。我们介绍了VisPlay，一种自我进化RL框架，使VLMs能够利用大量未标注的图像数据自主提升其推理能力。从单一基础VLM开始，VisPlay将模型分为两个相互作用的角色：一个图像条件提问者，生成具有挑战性但可回答的视觉问题，以及一个多模态推理者，生成银色回答。这些角色通过组相对策略优化（GRPO）共同训练，该方法结合了多样性和难度奖励，以平衡生成问题的复杂性与银色回答的质量。VisPlay在两个模型家族中高效扩展。当在Qwen2.5-VL和MiMo-VL上训练时，VisPlay在八个基准测试中实现了视觉推理、组合泛化和幻觉减少的一致改进，包括MM-Vet和MMMU，展示了向自我进化多模态智能的可扩展路径。项目页面可在https://bruno686.github.io/VisPlay/上访问。
英文摘要：
Reinforcement learning (RL) provides a principled framework for improving Vision-Language Models (VLMs) on complex reasoning tasks. However, existing RL approaches often rely on human-annotated labels or task-specific heuristics to define verifiable rewards, both of which are costly and difficult to scale. We introduce VisPlay, a self-evolving RL framework that enables VLMs to autonomously improve their reasoning abilities using large amounts of unlabeled image data. Starting from a single base VLM, VisPlay assigns the model into two interacting roles: an Image-Conditioned Questioner that formulates challenging yet answerable visual questions, and a Multimodal Reasoner that generates silver responses. These roles are jointly trained with Group Relative Policy Optimization (GRPO), which incorporates diversity and difficulty rewards to balance the complexity of generated questions with the quality of the silver answers. VisPlay scales efficiently across two model families. When trained on Qwen2.5-VL and MiMo-VL, VisPlay achieves consistent improvements in visual reasoning, compositional generalization, and hallucination reduction across eight benchmarks, including MM-Vet and MMMU, demonstrating a scalable path toward self-evolving multimodal intelligence. The project page is available at https://bruno686.github.io/VisPlay/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
