InSight: 通过可引导的VLA实现自主技能获取
InSight: Self-Guided Skill Acquisition via Steerable VLAs
AI总结：
提出InSight框架，通过将VLA模型在基本动作层面变得可引导，实现自主技能获取，包括自动分割演示为基本动作和VLM引导的数据飞轮，无需人类演示即可学习新技能。
摘要：
视觉-语言-动作（VLA）模型可以从演示中学习操作技能，但其能力受限于训练数据中的技能。我们提出InSight，一个通过使VLA在基本动作层面（例如，“将夹爪移动到碗边”、“向上抬起”、“倾倒瓶子”）变得可引导，从而解锁自主技能获取的框架。InSight包括两个主要阶段：（1）一个自动分割流程，通过VLM计划分解和末端执行器姿态将演示分割为带标签的基本动作，以实现VLA基本动作的可引导性；（2）一个VLM引导的数据飞轮，识别完成新任务所需的缺失基本动作，自主尝试使用VLM提出的低级控制来演示缺失的基本动作，并自动标记、存储和整合成功的演示到VLA训练集中。我们在仿真和真实世界操作任务中评估InSight，包括翻转方块、关闭抽屉、清扫、扭转和倾倒，无需任何这些目标技能的人类演示。一旦学习到，这些基本动作可以组合起来执行新的、长时域任务，无需额外的人类演示。我们的发现表明，基本动作的可引导性为VLA策略中的持续技能获取提供了实用基础。项目网站：此 https URL。
英文摘要：
Vision-language-action (VLA) models excel at robot manipulation via imitation learning, but adapting them to new tasks often requires additional human demonstrations, which can be costly or infeasible. Meanwhile, vision-language models (VLMs) offer semantic task understanding but lack the physical grounding required for execution. To bridge this gap, we present InSight, a framework for self-guided skill acquisition that uses a VLM to identify primitives missing from a VLA's repertoire, grounds the VLM's proposals through robot execution, and distills new primitives from successful rollouts into the VLA. Primitive steerability, the ability to execute and terminate primitives on command, enables the robot to reuse known primitives while collecting training data for missing primitives without requiring full-task human demonstrations for each new task. InSight has two stages: (1) a VLM automatically segments existing demonstrations into primitive-labeled trajectories to fine-tune a primitive-steerable VLA, and (2) the VLM plans a sequence of known primitives executed by the VLA and new primitives attempted by VLM-parameterized low-level controllers. New-primitive segments from successful task rollouts are added to the training data, and the VLA is retrained. The adapted VLA can then reliably execute new skills using the acquired primitives, without per-primitive VLM calls. We evaluate InSight on six simulated and real-world tasks with no human demonstrations of target skills, including block flipping, drawer closing, sweeping, twisting, and pouring. On hardware, acquired twisting and pouring skills achieve 92% and 96% success, versus 32% and 16% for a zero-shot CaP-X baseline. Composing both skills into a 14-primitive task achieves 80% success with no combined-task demonstrations. Project website: https://insight-vla.github.io/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
