视角至关重要:基于掩码自动编码器的动态视角优化用于视觉操控
Viewpoint Matters: Dynamically Optimizing Viewpoints with Masked Autoencoder for Visual Manipulation
AI总结:
MAE-Select通过动态选择最具信息量的视角,提升单摄像头机器人视觉操控性能,甚至超越多摄像头系统。
摘要:
机器人操控仍然面临挑战,模仿学习(IL)使机器人能够从专家示范中学习任务。当前IL方法通常依赖于固定摄像头设置,摄像头被手动放置在静态位置,对适应性和覆盖范围施加了显著限制。受人类主动感知的启发,人类会动态调整视角以捕捉最相关且最干净的信息,我们提出了MAE-Select,一种用于单摄像头机器人系统中主动视角选择的新框架。MAE-Select充分利用预训练的多视角掩码自动编码器表示,并在每个时间片段内动态选择下一个最具信息量的视角,而无需标注视角。广泛实验表明,MAE-Select增强了单摄像头系统的性能,在某些情况下甚至超越了多摄像头设置。该项目可在https://mae-select.github.io上获取。
英文摘要:
Robotic manipulation continues to be a challenge, and imitation learning (IL) enables robots to learn tasks from expert demonstrations. Current IL methods typically rely on fixed camera setups, where cameras are manually positioned in static locations, imposing significant limitations on adaptability and coverage. Inspired by human active perception, where humans dynamically adjust their viewpoint to capture the most relevant and least noisy information, we propose MAE-Select, a novel framework for active viewpoint selection in single-camera robotic systems. MAE-Select fully leverages pre-trained multi-view masked autoencoder representations and dynamically selects the next most informative viewpoint at each time chunk without requiring labeled viewpoints. Extensive experiments demonstrate that MAE-Select improves the capabilities of single-camera systems and, in some cases, even surpasses multi-camera setups. The project will be available at https://mae-select.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。