基于单次人类视频引导的视觉-语言-动作模型适配
One-shot Human Video-Guided Adaptation for Vision-Language-Action Models
AI总结：
提出CapPro框架，仅用单次人类演示视频即可让VLA策略适配新操作任务，实验显示其提升了未见过目标组的平均性能且保留源任务性能，验证了人类视频指导的有效性。
摘要：
视觉-语言-动作（VLA）模型近期通过利用大规模机器人数据集和基础模型实现了通用机器人操作的显著进展，但其快速适配训练分布之外的新操作任务的能力仍然有限。现有方法通常需要特定任务的机器人演示和额外训练，这既耗费人力又占用计算资源，限制了VLA适配在开放世界场景中的可扩展性。为减少对目标任务机器人数据的依赖，我们提出CapPro框架，该框架仅使用单次人类演示视频即可让VLA策略适配此前未见过的操作任务。CapPro包含两个组件：第一，基于阶段感知的人类视频表示模块，通过学习的视觉令牌选择构建紧凑的视频表示，并在阶段字幕监督下学习与任务相关的查询表示；第二，视频条件适配模块，将紧凑视频和查询衍生表示共同整合到预训练VLA策略的动作条件上下文中。该设计引入了特定任务的指导，同时在很大程度上保留了原始VLA的动作生成主干。通过这种方式，人类演示提供了特定任务的执行指导，而非需要模仿的与实体对齐的轨迹，无需目标任务机器人演示或目标任务微调。在源任务和未见过的目标任务上开展的实验表明，CapPro提升了未见过的目标组的平均性能，同时在源任务上保持了相当的性能。这些结果证明了人类视频指导在跨任务VLA适配中的有效性。
英文摘要：
Vision-language-action (VLA) models have recently achieved remarkable progress by leveraging large-scale robot datasets and foundation models for general-purpose robot manipulation. However, their ability to rapidly adapt to novel manipulation tasks beyond the training distribution remains limited. Existing approaches often require task-specific robot demonstrations and additional training, which are labor- and compute-intensive, limiting the scalability of VLA adaptation to open-world scenarios. To reduce this dependence on target-task robot data, we propose CapPro, a framework that enables a VLA policy to adapt to previously unseen manipulation tasks using only a single human demonstration video. CapPro consists of two components. First, a phase-aware human video representation module constructs a compact video representation through learned visual token selection and learns task-relevant query representations under phase-caption supervision. Second, a video-conditioned adaptation module jointly incorporates the compact video and query-derived representations into the action-conditioning context of the pretrained VLA policy. % This design introduces task-specific guidance while largely preserving the original VLA action-generation backbone. In this way, the human demonstration provides task-specific execution guidance rather than an embodiment-aligned trajectory to imitate, without requiring target-task robot demonstrations or target-task fine-tuning. Experiments across source and unseen target tasks show that CapPro improves average performance across unseen target groups while maintaining comparable performance on the source tasks. These results demonstrate the effectiveness of human-video guidance for cross-task VLA adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
