DynaRend：面向机器人操作的基于掩码未来渲染的3D动力学学习方法
DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation
AI总结：
针对机器人操作策略泛化性不足的问题，提出DynaRend框架，通过可微体渲染的掩码重建与未来预测学习3D动力学三平面特征，在基准测试与真实实验中均提升了操作成功率与泛化性。
摘要：
由于真实世界多样化训练数据的稀缺，学习具备泛化性的机器人操作策略仍是一项核心挑战。尽管近期研究尝试通过自监督表示学习缓解这一问题，但多数方法要么依赖掩码图像建模这类2D视觉预训练范式（主要关注静态语义或场景几何），要么采用侧重2D动力学的大规模视频预测模型，因而无法联合学习有效操作所需的几何、语义与动力学特征。\n本文提出DynaRend——一种表示学习框架，通过可微体渲染的掩码重建与未来预测机制，学习具备3D感知与动力学信息的三平面特征。DynaRend在多视角RGB-D视频数据上预训练，可在统一的三平面表示中联合捕捉空间几何、未来动力学与任务语义。学习到的表示可通过动作值图预测有效迁移至下游机器人操作任务。\n我们在RLBench、Colosseum两个挑战性基准以及真实机器人实验中评估了DynaRend，结果表明该方法在策略成功率、对环境扰动的泛化性以及多样操作任务的真实场景适用性上均有显著提升。
英文摘要：
Learning generalizable robotic manipulation policies remains a key challenge due to the scarcity of diverse real-world training data. While recent approaches have attempted to mitigate this through self-supervised representation learning, most either rely on 2D vision pretraining paradigms such as masked image modeling, which primarily focus on static semantics or scene geometry, or utilize large-scale video prediction models that emphasize 2D dynamics, thus failing to jointly learn the geometry, semantics, and dynamics required for effective manipulation. In this paper, we present DynaRend, a representation learning framework that learns 3D-aware and dynamics-informed triplane features via masked reconstruction and future prediction using differentiable volumetric rendering. By pretraining on multi-view RGB-D video data, DynaRend jointly captures spatial geometry, future dynamics, and task semantics in a unified triplane representation. The learned representations can be effectively transferred to downstream robotic manipulation tasks via action value map prediction. We evaluate DynaRend on two challenging benchmarks, RLBench and Colosseum, as well as in real-world robotic experiments, demonstrating substantial improvements in policy success rate, generalization to environmental perturbations, and real-world applicability across diverse manipulation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
