ERMV：编辑4D机器人多视图图像以增强具身智能体
ERMV: Editing 4D Robotic Multi-view images to enhance embodied agents
AI总结：
针对机器人4D多视图操作数据稀缺问题，本文提出ERMV框架，通过EMA-Attn、稀疏STT模块和MLLM反馈干预实现一致、低成本的整序列编辑，显著增强VLA模型的鲁棒性与泛化性。
摘要：
机器人模仿学习依赖4D多视图时序图像。然而，高昂的数据采集成本和高质量数据的稀缺严重制约了视觉-语言-动作（VLA）模型等具身智能策略的泛化与应用。数据增强是克服数据稀缺的有力策略，但目前缺乏用于编辑操作任务4D多视图时序图像的方法。因此，我们提出ERMV（Editing Robotic Multi-View 4D data，编辑机器人多视图4D数据），这是一种新颖的数据增强框架，可基于单帧编辑和机器人状态条件高效编辑整个多视图序列。该任务面临三大核心挑战：（1）在动态视图和长时间跨度内保持几何与外观一致性；（2）以低计算成本扩展工作窗口；（3）确保机械臂等关键对象的语义完整性。ERMV通过一系列创新应对这些挑战。首先，为确保运动模糊中的时空一致性，我们引入新颖的极线运动感知注意力（EMA-Attn）机制，在施加几何约束之前学习由运动引起的像素位移。其次，为最大化编辑工作窗口，ERMV率先提出稀疏时空（STT）模块，该模块解耦时间和空间视图，并通过对视图进行稀疏采样来重构单帧多视图问题，从而降低计算需求。第三，为缓解误差累积，我们引入反馈干预机制，使用多模态大语言模型（MLLM）检查编辑不一致性，并仅在必要时请求有针对性的专家指导。大量实验表明，经ERMV增强的数据在仿真和真实世界环境中均显著提升了VLA模型的鲁棒性和泛化能力。
英文摘要：
Robot imitation learning relies on 4D multi-view sequential images. However, the high cost of data collection and the scarcity of high-quality data severely constrain the generalization and application of embodied intelligence policies like Vision-Language-Action (VLA) models. Data augmentation is a powerful strategy to overcome data scarcity, but methods for editing 4D multi-view sequential images for manipulation tasks are currently lacking. Thus, we propose ERMV (Editing Robotic Multi-View 4D data), a novel data augmentation framework that efficiently edits an entire multi-view sequence based on single-frame editing and robot state conditions. This task presents three core challenges: (1) maintaining geometric and appearance consistency across dynamic views and long time horizons; (2) expanding the working window with low computational costs; and (3) ensuring the semantic integrity of critical objects like the robot arm. ERMV addresses these challenges through a series of innovations. First, to ensure spatio-temporal consistency in motion blur, we introduce a novel Epipolar Motion-Aware Attention (EMA-Attn) mechanism that learns pixel shift caused by movement before applying geometric constraints. Second, to maximize the editing working window, ERMV pioneers a Sparse Spatio-Temporal (STT) module, which decouples the temporal and spatial views and remodels a single-frame multi-view problem through sparse sampling of the views to reduce computational demands. Third, to alleviate error accumulation, we incorporate a feedback intervention Mechanism, which uses a Multimodal Large Language Model (MLLM) to check editing inconsistencies and request targeted expert guidance only when necessary. Extensive experiments demonstrate that ERMV-augmented data significantly boosts the robustness and generalization of VLA models in both simulated and real-world environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
