FoveaVLA：用中央凹注视动作替代视觉语言动作模型（VLA）中的腕部相机
FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs
AI总结：
FoveaVLA是一种用中央凹注视动作替代腕部相机的VLA模型，基于π₀.₅构建，在四个真实任务套件中，其单、双相机配置的操作成功率均显著高于配备腕部相机的基线模型，且在含干扰项等任务上表现更优。
摘要：
我们提出FoveaVLA，这是一种视觉语言动作（Vision-Language-Action，VLA）模型，它用学习得到的中央凹注视动作替代了腕部相机。腕部相机能提供近距离细节，但视野仅局限于末端执行器周围；外部相机可观测整个场景，但策略网络接收到的是低分辨率图像，因此会丢失操作所需的精细细节。FoveaVLA通过在每个动作块之外，额外预测下一次策略查询时需要观测的交互区域来恢复这些细节。随后，策略网络会接收从原始分辨率图像中裁剪出的这些区域的图像，以及完整的场景视图。我们在π₀.₅的基础上构建了FoveaVLA，分为单相机和双相机两种配置，分别使用1台或2台外部立体相机，输入为全分辨率和中央凹分辨率的RGB及深度数据，并在带有RoboInter交互标注的DROID数据集上进行后训练。在四个真实世界任务套件中，FoveaVLA单相机配置使用1台外部RGB-D视图实现了70.8%的成功率，双相机配置达到77.1%，而配备腕部相机的π₀.₅基线模型成功率为52.1%。两种配置在普通抓取放置任务上与基线表现相当，且在含干扰项的任务、布料操作及远距离物体操作的成功率上均有提升。
英文摘要：
We present FoveaVLA, a Vision-Language-Action (VLA) model that replaces the wrist camera with learned foveation actions. Wrist cameras provide close-up detail but restrict the view to the end-effector's surroundings. External cameras see the whole scene, but the policy receives them at low resolution, so the fine detail needed for manipulation is lost. FoveaVLA recovers this detail by predicting, alongside each action chunk, the interaction region to observe at the next policy query. The policy then receives crops of these regions, taken from the original-resolution images, together with the full views. We build FoveaVLA on $\pi_{0.5}$ in two configurations, Single and Dual, using one or two external stereo cameras with full and foveal RGB and depth inputs, and post-train both on DROID with RoboInter interaction annotations. Across four real-world task suites, FoveaVLA-Single achieves 70.8% success with one external RGB-D view, and FoveaVLA-Dual achieves 77.1%, versus 52.1% for the wrist-equipped $\pi_{0.5}$. Both match the baseline on plain pick-and-place and improve success with distractors, cloth manipulation, and far-away objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
