arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2603.15432cs.CV

Gym-V: 一个统一的视觉环境系统用于代理视觉研究

Gym-V: A Unified Vision Environment System for Agentic Vision Research

ModalMinds

Fanqing Meng, Lingxiao Du, Jiawei Gu, Jiaqi Liao, Linjie Li, Zijian Wu, Xiangyan Liu, Ziqi Zhao, Mengkang Hu, Zichen Liu, Jiaheng Zhang, Michael Qizhe Shieh

AI总结:

Gym-V通过统一的179个程序生成视觉环境,解决了视觉代理研究中缺乏标准化基础设施的问题,发现观察支架比RL算法选择更关键,且跨领域迁移实验显示多样化训练能广泛泛化。

摘要:

随着代理系统越来越多地依赖可验证奖励的强化学习,标准化的『gym』基础设施已成为快速迭代、可重复性和公平比较的关键。视觉代理缺乏此类基础设施,限制了对驱动其学习因素和当前模型不足的系统研究。我们介绍了Gym-V,一个包含10个领域179个程序生成视觉环境的统一平台,具有可控难度,使之前在碎片化工具包中不可行的受控实验成为可能。使用它,我们发现观察支架比RL算法选择对训练成功更为关键,标题和游戏规则决定学习是否成功。跨领域迁移实验进一步显示,多样化任务类别训练能广泛泛化,而狭窄训练可能导致负迁移,多轮交互放大了所有这些影响。Gym-V作为训练环境和评估工具包的便利基础发布,旨在加速未来代理视觉语言模型的研究。

英文摘要:

As agentic systems increasingly rely on reinforcement learning from verifiable rewards, standardized ``gym'' infrastructure has become essential for rapid iteration, reproducibility, and fair comparison. Vision agents lack such infrastructure, limiting systematic study of what drives their learning and where current models fall short. We introduce \textbf{Gym-V}, a unified platform of 179 procedurally generated visual environments across 10 domains with controllable difficulty, enabling controlled experiments that were previously infeasible across fragmented toolkits. Using it, we find that observation scaffolding is more decisive for training success than the choice of RL algorithm, with captions and game rules determining whether learning succeeds at all. Cross-domain transfer experiments further show that training on diverse task categories generalizes broadly while narrow training can cause negative transfer, with multi-turn interaction amplifying all of these effects. Gym-V is released as a convenient foundation for training environments and evaluation toolkits, aiming to accelerate future research on agentic VLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑