ExpReS-VLA: 通过经验回放和检索专门化视觉-语言-动作模型
ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval
AI总结:
ExpReS-VLA通过经验回放和检索增强,提升VLA模型在特定任务中的适应能力与性能。
摘要:
视觉-语言-动作(VLA)模型如OpenVLA在机器人操作任务中展示了出色的零样本泛化能力,但在特定部署环境中难以适应,因为一致的高绩效在有限任务集上比广泛的泛化更重要。我们提出了ExpReS-VLA,一种通过经验回放和检索增强的专门化VLA方法,该方法能够在设备上快速适应预训练的VLA到目标领域,同时通过压缩经验回放和检索增强生成防止灾难性遗忘。我们的方法通过存储从OpenVLA冻结视觉主干中提取的嵌入向量来保持内存高效的缓冲区,与原始图像-动作对相比,存储需求减少了97%。在部署过程中,ExpReS-VLA使用余弦相似度检索k个最相似的过去经验以增强训练批次,同时优先级经验回放缓冲区保存最近成功的轨迹。为了利用失败尝试,我们引入了阈值混合对比损失(THCL),使模型能够从成功和失败的演示中学习。在LIBERO基准测试中,ExpReS-VLA在空间推理任务上从82.6%提升到93.1%,在长时间任务上从61%提升到72.3%,在包括π₀(+3.2分)和OpenVLA-OFT(+1.7分)在内的各种架构上均取得进展。在五个任务上的物理机器人实验中,ExpReS-VLA在分布内和分布外条件下均实现了98%的成功率,相比朴素微调的84.7%和32%显著提升。使用12个演示在单个RTX 5090上完成适应仅需31秒。
英文摘要:
Vision-Language-Action (VLA) models like OpenVLA demonstrate impressive zero-shot generalization across robotic manipulation tasks but struggle to adapt to specific deployment environments where consistent high performance on a limited set of tasks is more valuable than broad generalization. We present EXPierence replayed, REtrieval augmented, Specialized VLA (ExpReS-VLA), a method that enables rapid on-device adaptation of pre-trained VLAs to target domains while preventing catastrophic forgetting through compressed experience replay and retrieval-augmented generation. Our approach maintains a memory-efficient buffer by storing extracted embeddings from OpenVLA's frozen vision backbone, reducing storage requirements by 97% compared to raw image-action pairs. During deployment, ExpReS-VLA retrieves the $k$ most similar past experiences using cosine similarity to augment training batches, while a prioritized experience replay buffer preserves recently successful trajectories. To leverage failed attempts, we introduce Thresholded Hybrid Contrastive Loss (THCL), enabling the model to learn from both successful and unsuccessful demonstrations. Experiments on the LIBERO benchmark show improvements from 82.6% to 93.1% on spatial reasoning and 61% to 72.3% on long-horizon tasks over base OpenVLA, with gains across architectures including $π_0$ (+3.2 points) and OpenVLA-OFT (+1.7 points). Physical robot experiments across five tasks demonstrate 98% success on both in-distribution and out-of-distribution conditions, improving from 84.7% and 32% respectively for naive fine-tuning. Adaptation completes in 31 seconds using 12 demonstrations on a single RTX 5090.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。