PokeVLA：通过全面的世界知识指导赋能便携式视觉-语言-动作模型
PokeVLA: Empowering Pocket-Sized Vision-Language-Action Model with Comprehensive World Knowledge Guidance
AI总结：
本文提出PokeVLA，一种轻量但强大的具身体验模型，通过预训练视觉语言模型和多视角目标感知学习提升机器人操作效率与空间认知能力。
摘要：
近期视觉-语言-动作（VLA）模型的进步为机器人操作开辟了新途径，但现有方法在效率和高层知识与空间意识方面存在局限。为解决这些问题，我们提出了PokeVLA，一种轻量但强大的基础模型，有效将视觉语言理解融入动作学习。我们的框架引入了两阶段训练范式：首先，在包含空间定位、可能性和具身推理任务的240万样本多模态数据集上预训练紧凑的视觉语言模型（PokeVLM）；其次，通过多视角目标感知语义学习、几何对齐和新的动作专家将操作相关表示注入动作空间。大量实验表明，PokeVLA在LIBERO-Plus基准和现实世界部署中表现出最先进的性能，优于现有基线在成功率和在多样扰动下的鲁棒性。为促进可重复性和社区进展，我们将开源我们的代码、模型权重和定制预训练数据集的脚本。项目页面：https://getterupper.github.io/PokeVLA
英文摘要：
Recent advances in Vision-Language-Action (VLA) models have opened new avenues for robot manipulation, yet existing methods exhibit limited efficiency and a lack of high-level knowledge and spatial awareness. To address these challenges, we propose PokeVLA, a lightweight yet powerful foundation model for embodied manipulation that effectively infuses vision-language understanding into action learning. Our framework introduces a two-stage training paradigm: first, we pre-train a compact vision-language model (PokeVLM) on a curated multimodal dataset of 2.4M samples encompassing spatial grounding, affordance, and embodied reasoning tasks; second, we inject manipulation-relevant representations into the action space through multi-view goal-aware semantics learning, geometry alignment, and a novel action expert. Extensive experiments demonstrate state-of-the-art performance on the LIBERO-Plus benchmark and in real-world deployment, outperforming comparable baselines in success rate and robustness under diverse perturbations. To foster reproducibility and community progress, we will open-source our code, model weights, and the scripts for the curated pre-training dataset. Project page: https://getterupper.github.io/PokeVLA
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
