UAM：VL A训练中遗忘的双流视角
UAM: A Dual-Stream Perspective on Forgetting in VLA Training
AI总结：
本文提出UAM模型，通过双流架构解决VL A训练中因单一编码器导致的多模态能力下降问题，展示了通过架构分离而非冻结权重或辅助数据可实现语义保留，并在多种任务中取得高成功率。
摘要：
视觉-语言-动作（VLA）模型通常通过在动作数据上微调预训练的视觉-语言模型（VLM）来构建。然而，我们证明这种标准方法系统性地削弱了VLM的多模态能力，这种副作用我们称之为‘具身税’。但VL A是否必须遗忘？受生物视觉双流组织的启发，我们将这种退化归因于结构性瓶颈：当前VL A要求单一编码器同时支持语言基础语义和控制相关的视觉特征，而生物视觉将识别与视觉运动控制分为不同的路径。基于此观点，我们提出了统一动作模型（UAM），添加了一个平行的背侧专家，作为大脑背侧通路的类比。为了使背侧专家成为有效的第二路径并减少对VLM的控制学习负担，我们从预训练的生成模型中初始化它，并用中层推理目标进行训练，该目标预测视觉动态。这种设计使我们能够仅用动作数据端到端地训练整个VLA：无需参数冻结、无需梯度停止、无需辅助VL共训练，UAM保留了超过95%的底层VLM的多模态能力，同时在多种任务中取得了最高平均成功率，包括未见物体、新物体-目标组合和指令变化等探测分布外泛化的任务。这些结果表明，VL A中的语义保留可以从架构分离本身产生，而非通过冻结权重或辅助数据重放，并且这种保留的语义能力可以自然地从VLM转移到动作中的语义泛化。
英文摘要：
Vision--language--action (VLA) models are typically built by fine-tuning a pretrained vision--language model (VLM) on action data. However, we show that this standard recipe systematically erodes the VLM's multimodal competence, a side effect we call the embodiment tax. But do VLAs have to forget? Inspired by the two-stream organization of biological vision, we trace this degradation to a structural bottleneck: current VLAs ask a single encoder to support both language-grounded semantics and control-relevant visual features, whereas biological vision separates recognition and visuomotor control into distinct pathways. Building on this view, we propose the Unified Action Model (UAM), which adds a parallel Dorsal Expert, an analog of the brain's dorsal pathway. To make the Dorsal Expert an effective second pathway and reduce the control-learning burden on the VLM, we initialize it from a pretrained generative model and train it with a mid-level reasoning objective that predicts visual dynamics. This design allows us to train the whole VLA end-to-end on action data alone: with no parameter freezing, no gradient stopping, and no auxiliary VL co-training, UAM retains over $95\%$ of the underlying VLM's multimodal capability and at the same time achieves the highest average success rate among baselines on a variety of manipulation tasks that probe out-of-distribution generalization, including unseen objects, novel object--target compositions, and instruction variation. Together, these results suggest that semantic preservation in VLAs can emerge from architectural separation itself, rather than being enforced by frozen weights or auxiliary data replay, and that this preserved semantic capability can naturally transfer from VLMs to semantic generalization in actions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
