AnyCamVLA: 零样本相机适应用于视角鲁棒的视觉-语言-动作模型
AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models
AI总结：
AnyCamVLA通过零样本相机适应提升视觉-语言-动作模型在视角变化下的鲁棒性，适用于任何RGB策略。
摘要：
尽管在机器人操作中视觉-语言-动作模型（VLAs）取得了显著进展，但这些大型预训练模型需要微调才能部署到特定环境中。这些微调后的模型对相机视角变化非常敏感，而这种变化在非结构化环境中经常发生。在本文中，我们提出了一种无需额外演示数据、策略微调或架构修改的零样本相机适应框架。我们的关键思想是实时地虚拟调整测试时的相机观察以匹配训练时的相机配置。为此，我们使用了一个最近的前馈新视角合成模型，该模型输出高质量的目标视角图像，能够处理内外参数。这种即插即用的方法保留了VLAs的预训练能力，并适用于任何基于RGB的策略。通过在LIBERO基准上的广泛实验，我们的方法在使用数据增强进行策略微调或额外3D-aware特征进行视觉输入的基线方法上表现一致更优。我们进一步验证了我们的方法在现实世界机器人操作场景中持续增强了视角鲁棒性，包括具有变化的相机外参数、内参数和自由移动手持相机的设置。
英文摘要：
Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
