M³A策略:通过光度重渲染的可变材料操控增强策略
$\mathbf{M^3A}$ Policy: Mutable Material Manipulation Augmentation Policy through Photometric Re-rendering
AI总结:
M³A策略通过光度重渲染生成多样化演示,提升机器人跨材料操控的泛化能力。
摘要:
材料泛化对于现实世界中的机器人操控至关重要,其中机器人必须与表现出多样视觉和物理特性的物体互动。这一挑战在玻璃、金属或其他材料制成的物体上尤为突出,因为透明或反射表面引入了严重的分布外变化。现有方法要么依赖于模拟器中的模拟材料并进行sim-to-real转移,但受到显著的视觉域差距限制;要么依赖于收集大量现实世界演示,这成本高、耗时且仍无法覆盖各种材料。为克服这些限制,我们采用计算摄影技术,引入可变材料操控增强(M³A),一个统一的框架,利用材料的物理特性作为光传输捕捉的光源重新渲染。核心思想简单而强大:给定一个现实世界的演示,我们通过光度重渲染场景,生成具有不同材料特性的多样化高现实感演示集。这种增强有效解耦了任务特定的操控技能与表面外观,使策略能够在不额外收集数据的情况下跨材料泛化。为系统评估这一能力,我们构建了首个全面的多材料操控基准,涵盖仿真和现实环境。大量实验表明,M³A策略显著提高了跨材料泛化能力,使三个现实任务的平均成功率提高了58.03%,并在之前未见过的材料上表现出鲁棒性能。
英文摘要:
Material generalization is essential for real-world robotic manipulation, where robots must interact with objects exhibiting diverse visual and physical properties. This challenge is particularly pronounced for objects made of glass, metal, or other materials whose transparent or reflective surfaces introduce severe out-of-distribution variations. Existing approaches either rely on simulated materials in simulators and perform sim-to-real transfer, which is hindered by substantial visual domain gaps, or depend on collecting extensive real-world demonstrations, which is costly, time-consuming, and still insufficient to cover various materials. To overcome these limitations, we resort to computational photography and introduce Mutable Material Manipulation Augmentation (M$^3$A), a unified framework that leverages the physical characteristics of materials as captured by light transport for photometric re-rendering. The core idea is simple yet powerful: given a single real-world demonstration, we photometrically re-render the scene to generate a diverse set of highly realistic demonstrations with different material properties. This augmentation effectively decouples task-specific manipulation skills from surface appearance, enabling policies to generalize across materials without additional data collection. To systematically evaluate this capability, we construct the first comprehensive multi-material manipulation benchmark spanning both simulation and real-world environments. Extensive experiments show that the M$^3$A policy significantly enhances cross-material generalization, improving the average success rate across three real-world tasks by 58.03\%, and demonstrating robust performance on previously unseen materials.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。