VLM4VLA：重新审视视觉-语言-动作模型中的视觉-语言模型
VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models
AI总结：
本文通过VLM4VLA最小适配管道，系统研究视觉-语言模型（VLM）的选择和能力如何影响下游视觉-语言-动作（VLA）策略性能，发现VLM通用能力无法预测下游任务表现，且视觉模块是性能瓶颈。
摘要：
视觉-语言-动作（VLA）模型将预训练的大型视觉-语言模型（VLM）集成到其策略主干中，因其有前景的泛化能力而受到广泛关注。本文重新审视了一个基本但很少被系统研究的问题：VLM的选择和能力如何转化为下游VLA策略的性能？我们引入了VLM4VLA，一个最小适配管道，仅使用少量新的可学习参数将通用VLM转换为VLA策略，以实现公平高效的比较。尽管简单，VLM4VLA被证明与更复杂的网络设计相比具有惊人的竞争力。通过在三个基准上的各种下游任务进行广泛的实证研究，我们发现虽然VLM初始化比从头训练提供了一致的优势，但VLM的通用能力并不能很好地预测其下游任务性能。这挑战了常见的假设，表明标准VLM能力对于有效的具身控制是必要但不充分的。我们进一步通过微调VLM在七个辅助具身任务（例如，具身问答、视觉指向、深度估计）上研究特定具身能力的影响。与直觉相反，提高VLM在特定具身技能上的性能并不能保证更好的下游控制性能。最后，模态级别的消融实验确定VLM中的视觉模块（而非语言组件）是主要的性能瓶颈。我们证明，即使在下游微调期间编码器保持冻结，向VLM的视觉编码器注入控制相关的监督也能带来一致的收益。这隔离了当前VLM预训练目标与具身动作规划需求之间持续的领域差距。
英文摘要：
Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities. This paper revisits a fundamental yet seldom systematically studied question: how VLM choice and competence translate to downstream VLA policies performance? We introduce VLM4VLA, a minimal adaptation pipeline that converts general-purpose VLMs into VLA policies using only a small set of new learnable parameters for fair and efficient comparison. Despite its simplicity, VLM4VLA proves surprisingly competitive with more sophisticated network designs. Through extensive empirical studies on various downstream tasks across three benchmarks, we find that while VLM initialization offers a consistent benefit over training from scratch, a VLM's general capabilities are poor predictors of its downstream task performance. This challenges common assumptions, indicating that standard VLM competence is necessary but insufficient for effective embodied control. We further investigate the impact of specific embodied capabilities by fine-tuning VLMs on seven auxiliary embodied tasks (e.g., embodied QA, visual pointing, depth estimation). Contrary to intuition, improving a VLM's performance on specific embodied skills does not guarantee better downstream control performance. Finally, modality-level ablations identify the visual module in VLM, rather than the language component, as the primary performance bottleneck. We demonstrate that injecting control-relevant supervision into the vision encoder of the VLM yields consistent gains, even when the encoder remains frozen during downstream fine-tuning. This isolates a persistent domain gap between current VLM pretraining objectives and the requirements of embodied action-planning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
