TwinBrainVLA: 通过非对称Transformer混合释放通用视觉语言模型在具身任务中的潜力
TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers
AI总结：
TwinBrainVLA通过非对称Transformer混合机制，利用冻结的通用视觉语言模型和可训练的专家路径，实现机器人任务中的具身智能提升。
摘要：
视觉语言动作（VLA）模型的核心前提是利用预训练的视觉语言模型（VLM）的广泛通用能力来实现通用具身智能。然而，标准的机器人微调不可避免地会破坏预训练的特征空间，导致"灾难性遗忘"，从而损害我们希望利用的通用视觉理解。为了有效利用未受破坏的VLM通用能力来执行机器人任务，我们提出了TwinBrainVLA，该架构协调两个同构的VLM路径：一个冻结的通用路径（也称为"左脑"）和一个可训练的专家路径（也称为"右脑"）。我们的架构使用非对称Transformer混合（AsyMoT）机制，使右脑能够动态查询并融合来自左脑的完整语义知识与本体状态。融合的表示条件了一个流匹配动作专家，以实现精确的连续控制。在SimplerEnv和RoboCasa基准测试中，实验证明，通过显式保留通用能力，TwinBrainVLA在复杂操作任务中相对于基线模型实现了显著的性能提升。
英文摘要：
The fundamental premise of Vision-Language-Action (VLA) models is to harness the extensive general capabilities of pre-trained Vision-Language Models (VLMs) for generalized embodied intelligence. However, standard robotic fine-tuning inevitably disrupts the pre-trained feature space, leading to "catastrophic forgetting" that compromises the general visual understanding we aim to leverage. To effectively utilize the uncorrupted general capabilities of VLMs for robotic tasks, we propose TwinBrainVLA, which coordinates two isomorphic VLM pathways: a frozen generalist (also called "Left Brain") and a trainable specialist (also called "Right Brain"). Our architecture utilizes a Asymmetric Mixture-of-Transformers (AsyMoT) mechanism, enabling the Right Brain to dynamically query and fuse intact semantic knowledge from the Left Brain with proprioceptive states. This fused representation conditions a flow-matching action expert for precise continuous control. Empirical results on SimplerEnv and RoboCasa benchmarks demonstrate that by explicitly retaining general capabilities, TwinBrainVLA achieves substantial performance gains over baseline models in complex manipulation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
